* [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language
@ 2020-08-26 15:14 Cristian Dumitrescu
2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu
` (39 more replies)
0 siblings, 40 replies; 329+ messages in thread
From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw)
To: dev
This patch set introduces a new pipeline type that combines the DPDK
performance with the flexibility of the P4-16 language[1]. The new API
can be used either by itself to code a complete software switch (SWX)
or data plane app, or in combination with the open-source P4 compiler
P4C [2], potentially acting as a P4C back-end, thus allowing the P4
programs to be translated to the DPDK API and run on multi-core CPUs.
Main new features:
* Nothing is hard-wired, everything is dynamically defined: The packet
headers (i.e. protocols), the packet meta-data, the actions, the
tables and the pipeline itself are dynamically defined instead of
having to be selected from a pre-defined set.
* Instructions: The actions and the life of the packet through the
pipeline are defined with instructions that manipulate the pipeline
objects mentioned above. The pipeline is the main function of the
packet program, with actions as subroutines triggered by the tables.
* Call external plugins: Extern objects and functions can be defined
to call functionality that cannot be efficiently implemented with
the existing pipeline-oriented instruction set, such as: special
error detecting/correcting codes, crypto, meters, stats arrays,
heuristics, etc.
* Better control plane interaction: Transaction-oriented table update
mechanism that supports multi-table atomic updates. Multiple tables
can be updated in a single step with only the before and after table
sets visible to the packets. Alignment with P4Runtime [3].
* Performance: Multiple packets are in-flight within the pipeline at
any moment. Each packet is owned by a different time-sharing thread
in run-to-completion, with the thread pausing before memory access
operations such as packet I/O and table lookup to allow the memory
prefetch to complete. The instructions are verified and translated
at initialization time with no run-time impact. The instructions are
also optimized to detect and "fuse" frequently used patterns into
vector-like instructions transparently to the user.
[1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf
[2] P4-16 compiler: https://github.com/p4lang/p4c
[3] P4Runtime specification:
https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf
Cristian Dumitrescu (40):
pipeline: add pipeline
pipeline: add input port
pipeline: add output port
pipeline: add headers and meta-data
pipeline: add extern objects and functions
pipeline: add action
pipeline: add tables
pipeline: add pipeline instructions
pipeline: add rx and extract instructions
pipeline: add tx and emit instructions
pipeline: add header validate and invalidate instructions
pipeline: add mov instruction
pipeline: add dma instruction
pipeline: introduce add instruction
pipeline: introduce sub instruction
pipeline: introduce ckadd instruction
pipeline: introduce cksub instruction
pipeline: introduce and instruction
pipeline: introduce or instruction
pipeline: introduce xor instruction
pipeline: introduce shl instruction
pipeline: introduce shr instruction
pipeline: introduce table instruction
pipeline: introduce extern instruction
pipeline: introduce jmp and return instructions
pipeline: add instruction verifier
pipeline: add instruction optimizer
pipeline: add pipeline query API
pipeline: add pipeline flush
pipeline: add instruction description
pipeline: add table update high level API
port: add ethernet device port
port: add source and sink ports
table: add exact match table
examples/pipeline: add new example application
examples/pipeline: add message passing mechanism
examples/pipeline: add configuration commands
examples/pipeline: add l2fwd example
examples/pipeline: add l2fwd with MAC swap example
examples/pipeline: add VXLAN encap example
examples/Makefile | 1 +
examples/meson.build | 1 +
examples/pipeline/Makefile | 85 +
examples/pipeline/cli.c | 1394 ++++
examples/pipeline/cli.h | 19 +
examples/pipeline/conn.c | 331 +
examples/pipeline/conn.h | 50 +
examples/pipeline/example_l2fwd.c | 125 +
examples/pipeline/example_l2fwd_macswp.c | 146 +
examples/pipeline/example_vxlan.c | 318 +
examples/pipeline/examples/l2fwd.cli | 25 +
examples/pipeline/examples/l2fwd_macswp.cli | 25 +
.../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +
examples/pipeline/examples/l2fwd_pcap.cli | 20 +
examples/pipeline/examples/packet.txt | 102 +
examples/pipeline/examples/vxlan.cli | 27 +
examples/pipeline/examples/vxlan.py | 71 +
examples/pipeline/examples/vxlan.txt | 16 +
examples/pipeline/examples/vxlan_pcap.cli | 22 +
examples/pipeline/main.c | 193 +
examples/pipeline/meson.build | 21 +
examples/pipeline/obj.c | 470 ++
examples/pipeline/obj.h | 131 +
examples/pipeline/thread.c | 549 ++
examples/pipeline/thread.h | 28 +
lib/librte_pipeline/Makefile | 5 +
lib/librte_pipeline/meson.build | 13 +-
lib/librte_pipeline/rte_pipeline_version.map | 43 +-
lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++
lib/librte_pipeline/rte_swx_ctl.h | 568 ++
lib/librte_pipeline/rte_swx_extern.h | 98 +
lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++
lib/librte_pipeline/rte_swx_pipeline.h | 685 ++
lib/librte_port/Makefile | 5 +
lib/librte_port/meson.build | 9 +-
lib/librte_port/rte_port_version.map | 5 +-
lib/librte_port/rte_swx_port.h | 202 +
lib/librte_port/rte_swx_port_ethdev.c | 313 +
lib/librte_port/rte_swx_port_ethdev.h | 54 +
lib/librte_port/rte_swx_port_source_sink.c | 335 +
lib/librte_port/rte_swx_port_source_sink.h | 57 +
lib/librte_table/Makefile | 3 +
lib/librte_table/meson.build | 7 +-
lib/librte_table/rte_swx_table.h | 295 +
lib/librte_table/rte_swx_table_em.c | 851 ++
lib/librte_table/rte_swx_table_em.h | 30 +
lib/librte_table/rte_table_version.map | 7 +
47 files changed, 16516 insertions(+), 8 deletions(-)
create mode 100644 examples/pipeline/Makefile
create mode 100644 examples/pipeline/cli.c
create mode 100644 examples/pipeline/cli.h
create mode 100644 examples/pipeline/conn.c
create mode 100644 examples/pipeline/conn.h
create mode 100644 examples/pipeline/example_l2fwd.c
create mode 100644 examples/pipeline/example_l2fwd_macswp.c
create mode 100644 examples/pipeline/example_vxlan.c
create mode 100644 examples/pipeline/examples/l2fwd.cli
create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli
create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli
create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli
create mode 100644 examples/pipeline/examples/packet.txt
create mode 100644 examples/pipeline/examples/vxlan.cli
create mode 100644 examples/pipeline/examples/vxlan.py
create mode 100644 examples/pipeline/examples/vxlan.txt
create mode 100644 examples/pipeline/examples/vxlan_pcap.cli
create mode 100644 examples/pipeline/main.c
create mode 100644 examples/pipeline/meson.build
create mode 100644 examples/pipeline/obj.c
create mode 100644 examples/pipeline/obj.h
create mode 100644 examples/pipeline/thread.c
create mode 100644 examples/pipeline/thread.h
create mode 100644 lib/librte_pipeline/rte_swx_ctl.c
create mode 100644 lib/librte_pipeline/rte_swx_ctl.h
create mode 100644 lib/librte_pipeline/rte_swx_extern.h
create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c
create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h
create mode 100644 lib/librte_port/rte_swx_port.h
create mode 100644 lib/librte_port/rte_swx_port_ethdev.c
create mode 100644 lib/librte_port/rte_swx_port_ethdev.h
create mode 100644 lib/librte_port/rte_swx_port_source_sink.c
create mode 100644 lib/librte_port/rte_swx_port_source_sink.h
create mode 100644 lib/librte_table/rte_swx_table.h
create mode 100644 lib/librte_table/rte_swx_table_em.c
create mode 100644 lib/librte_table/rte_swx_table_em.h
--
2.17.1
^ permalink raw reply [flat|nested] 329+ messages in thread* [dpdk-dev] [PATCH 01/40] pipeline: add pipeline 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 02/40] pipeline: add input port Cristian Dumitrescu ` (38 subsequent siblings) 39 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add improved pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 2 + lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 5 files changed, 162 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index cfbbd1828..32582db9e 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -21,8 +21,10 @@ EXPORT_MAP := rte_pipeline_version.map SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language 2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (40 more replies) 0 siblings, 41 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add pipeline pipeline: add input port pipeline: add output port pipeline: add headers and meta-data pipeline: add extern objects and functions pipeline: add action pipeline: add tables pipeline: add pipeline instructions pipeline: add rx and extract instructions pipeline: add tx and emit instructions pipeline: add header validate and invalidate instructions pipeline: add mov instruction pipeline: add dma instruction pipeline: introduce add instruction pipeline: introduce sub instruction pipeline: introduce ckadd instruction pipeline: introduce cksub instruction pipeline: introduce and instruction pipeline: introduce or instruction pipeline: introduce xor instruction pipeline: introduce shl instruction pipeline: introduce shr instruction pipeline: introduce table instruction pipeline: introduce extern instruction pipeline: introduce jmp and return instructions pipeline: add instruction description pipeline: add instruction verifier pipeline: add instruction optimizer pipeline: add pipeline query API pipeline: add pipeline flush pipeline: add table update high level API pipeline: add specification file port: add ethernet device port port: add source and sink ports table: add exact match table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example examples/Makefile | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 82 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/Makefile | 6 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/Makefile | 5 + lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/Makefile | 3 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 48 files changed, 17669 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (39 subsequent siblings) 40 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 2 + lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 5 files changed, 162 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index cfbbd1828..32582db9e 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -21,8 +21,10 @@ EXPORT_MAP := rte_pipeline_version.map SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (40 more replies) 0 siblings, 41 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX rx and extract instructions pipeline: add SWX tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX mov instruction pipeline: add SWX dma instruction pipeline: introduce SWX add instruction pipeline: introduce SWX sub instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX xor instruction pipeline: introduce SWX shl instruction pipeline: introduce SWX shr instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jmp and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example examples/meson.build | 1 + examples/pipeline/Makefile | 53 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 44 files changed, 17625 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-09 19:05 ` Stephen Hemminger 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (39 subsequent siblings) 40 siblings, 2 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 4 files changed, 160 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-09 19:05 ` Stephen Hemminger 2020-09-09 19:52 ` Dumitrescu, Cristian 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 1 sibling, 1 reply; 329+ messages in thread From: Stephen Hemminger @ 2020-09-09 19:05 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev On Tue, 8 Sep 2020 21:17:50 +0100 Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > +/* > + * Pipeline. > + */ > +struct rte_swx_pipeline { > + int build_done; > + int numa_node; > +}; > + > + Is int the right type for these. build_done seems like a boolean and numa_node should be unsigned? ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type 2020-09-09 19:05 ` Stephen Hemminger @ 2020-09-09 19:52 ` Dumitrescu, Cristian 2020-09-10 8:42 ` Bruce Richardson 0 siblings, 1 reply; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-09 19:52 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev Hi Stephen, > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Wednesday, September 9, 2020 8:06 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline > type > > On Tue, 8 Sep 2020 21:17:50 +0100 > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > +/* > > + * Pipeline. > > + */ > > +struct rte_swx_pipeline { > > + int build_done; > > + int numa_node; > > +}; > > + > > + > > Is int the right type for these. > build_done seems like a Boolean Isn't the difference between int and bool mostly cosmetic? AFAIK we don't have a hard rule in DPDK about bool vs. int. > and numa_node should be unsigned? All the functions in libnuma use int as the numa_node type, please see: man 3 numa. Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type 2020-09-09 19:52 ` Dumitrescu, Cristian @ 2020-09-10 8:42 ` Bruce Richardson 0 siblings, 0 replies; 329+ messages in thread From: Bruce Richardson @ 2020-09-10 8:42 UTC (permalink / raw) To: Dumitrescu, Cristian; +Cc: Stephen Hemminger, dev On Wed, Sep 09, 2020 at 07:52:39PM +0000, Dumitrescu, Cristian wrote: > Hi Stephen, > > > -----Original Message----- > > From: Stephen Hemminger <stephen@networkplumber.org> > > Sent: Wednesday, September 9, 2020 8:06 PM > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline > > type > > > > On Tue, 8 Sep 2020 21:17:50 +0100 > > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > > > +/* > > > + * Pipeline. > > > + */ > > > +struct rte_swx_pipeline { > > > + int build_done; > > > + int numa_node; > > > +}; > > > + > > > + > > > > Is int the right type for these. > > build_done seems like a Boolean > > Isn't the difference between int and bool mostly cosmetic? > AFAIK we don't have a hard rule in DPDK about bool vs. int. > > > and numa_node should be unsigned? > > All the functions in libnuma use int as the numa_node type, please see: man 3 numa. > Yes, we also have -1 in DPDK for an unspecified NUMA node, e.g. where a PCI device is just attached to the chipset rather than direct to the CPU socket. /Bruce ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-09 19:05 ` Stephen Hemminger @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (42 more replies) 1 sibling, 43 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V4 changes: * Spell check fixes. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX rx and extract instructions pipeline: add SWX tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX mov instruction pipeline: add SWX dma instruction pipeline: introduce SWX add instruction pipeline: introduce SWX sub instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX xor instruction pipeline: introduce SWX shl instruction pipeline: introduce SWX shr instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jmp and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example examples/meson.build | 1 + examples/pipeline/Makefile | 53 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 44 files changed, 17625 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (41 subsequent siblings) 42 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 4 files changed, 160 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (41 more replies) 0 siblings, 42 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V5 changes: * Upper case abberviations in some commit titles. * Added new example app in the MAINTAINERS file. * Absolutely no code changes. V4 changes: * Spell check fixes. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX rx and extract instructions pipeline: add SWX tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX mov instruction pipeline: add SWX dma instruction pipeline: introduce SWX add instruction pipeline: introduce SWX sub instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX xor instruction pipeline: introduce SWX shl instruction pipeline: introduce SWX shr instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jmp and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example MAINTAINERS | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 53 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 45 files changed, 17626 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:24 ` Stephen Hemminger 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (40 subsequent siblings) 41 siblings, 2 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 4 files changed, 160 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-23 18:24 ` Stephen Hemminger 2020-09-23 18:37 ` Dumitrescu, Cristian 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 1 sibling, 1 reply; 329+ messages in thread From: Stephen Hemminger @ 2020-09-23 18:24 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, thomas, david.marchand On Wed, 23 Sep 2020 19:06:05 +0100 Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > +/* > + * Pipeline. > + */ > +struct rte_swx_pipeline { > + int build_done; > + int numa_node; > +}; > + Nit, could build_done be a bool type? +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} The free() function in libc is defined to accept NULL as ok. Please remove the if() ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type 2020-09-23 18:24 ` Stephen Hemminger @ 2020-09-23 18:37 ` Dumitrescu, Cristian 2020-09-23 20:23 ` Stephen Hemminger 0 siblings, 1 reply; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-23 18:37 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev, thomas, david.marchand Hi Stephen, > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Wednesday, September 23, 2020 7:25 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev@dpdk.org; thomas@monjalon.net; david.marchand@redhat.com > Subject: Re: [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline > type > > On Wed, 23 Sep 2020 19:06:05 +0100 > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > +/* > > + * Pipeline. > > + */ > > +struct rte_swx_pipeline { > > + int build_done; > > + int numa_node; > > +}; > > + > > Nit, could build_done be a bool type? > As we discussed this in an earlier version of this patch set: Isn't the difference between int and bool mostly cosmetic? AFAIK we don't have a hard rule in DPDK about bool vs. int. IMO doing this change now it likely not going to add any value. > +void > +rte_swx_pipeline_free(struct rte_swx_pipeline *p) > +{ > + if (!p) > + return; > + > + free(p); > +} > > The free() function in libc is defined to accept NULL as ok. > Please remove the if() This is just the early function wrapper in patch 1 out of 41, mode code is added in this function by later patches that need the if statement. IMO this change will not add any value at all here. Thanks, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type 2020-09-23 18:37 ` Dumitrescu, Cristian @ 2020-09-23 20:23 ` Stephen Hemminger 0 siblings, 0 replies; 329+ messages in thread From: Stephen Hemminger @ 2020-09-23 20:23 UTC (permalink / raw) To: Dumitrescu, Cristian; +Cc: dev, thomas, david.marchand On Wed, 23 Sep 2020 18:37:19 +0000 "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com> wrote: > Hi Stephen, > > > -----Original Message----- > > From: Stephen Hemminger <stephen@networkplumber.org> > > Sent: Wednesday, September 23, 2020 7:25 PM > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > > Cc: dev@dpdk.org; thomas@monjalon.net; david.marchand@redhat.com > > Subject: Re: [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline > > type > > > > On Wed, 23 Sep 2020 19:06:05 +0100 > > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > > > +/* > > > + * Pipeline. > > > + */ > > > +struct rte_swx_pipeline { > > > + int build_done; > > > + int numa_node; > > > +}; > > > + > > > > Nit, could build_done be a bool type? > > > > As we discussed this in an earlier version of this patch set: > Isn't the difference between int and bool mostly cosmetic? > AFAIK we don't have a hard rule in DPDK about bool vs. int. > IMO doing this change now it likely not going to add any value. There is no policy and there probably doesn't need to be. Original code was written using BSD style, and BSD predates introduction of <stdbool.h>. Linux developers have been favoring bool. Purely a human thing, compilers just treat bool == int and allow assigning anything. Coverity or sparse might check though. > > +void > > +rte_swx_pipeline_free(struct rte_swx_pipeline *p) > > +{ > > + if (!p) > > + return; > > + > > + free(p); > > +} > > > > The free() function in libc is defined to accept NULL as ok. > > Please remove the if() > > This is just the early function wrapper in patch 1 out of 41, mode code is added in this function by later patches that need the if statement. IMO this change will not add any value at all here. Sure, makes sense. ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-23 18:24 ` Stephen Hemminger @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (42 more replies) 1 sibling, 43 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V6 changes: * Fixed issues in the example app Makefile. * Used rte_eth_link_speed_to_str() in the example app. * Added release notes update. V5 changes: * Upper case abberviations in some commit titles. * Added new example app in the MAINTAINERS file. * Absolutely no code changes. V4 changes: * Spell check fixes. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX rx and extract instructions pipeline: add SWX tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX mov instruction pipeline: add SWX dma instruction pipeline: introduce SWX add instruction pipeline: introduce SWX sub instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX xor instruction pipeline: introduce SWX shl instruction pipeline: introduce SWX shr instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jmp and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example doc: add new SWX pipeline type to release notes MAINTAINERS | 1 + doc/guides/rel_notes/release_20_11.rst | 11 + examples/meson.build | 1 + examples/pipeline/Makefile | 52 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 46 files changed, 17636 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (41 subsequent siblings) 42 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 4 files changed, 160 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu ` (42 more replies) 0 siblings, 43 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V7 changes: * Set copyright year to 2020 for all new files. * Fixed two Doxygen warnings. V6 changes: * Fixed issues in the example app Makefile. * Used rte_eth_link_speed_to_str() in the example app. * Added release notes update. V5 changes: * Upper case abberviations in some commit titles. * Added new example app in the MAINTAINERS file. * Absolutely no code changes. V4 changes: * Spell check fixes. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX Rx and extract instructions pipeline: add SWX Tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX move instruction pipeline: add SWX DMA instruction pipeline: introduce SWX add instruction pipeline: introduce SWX subtract instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX XOR instruction pipeline: introduce SWX SHL instruction pipeline: introduce SWX SHR instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jump and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example doc: add new SWX pipeline type to release notes MAINTAINERS | 1 + doc/guides/rel_notes/release_20_11.rst | 11 + examples/meson.build | 1 + examples/pipeline/Makefile | 52 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 46 files changed, 17636 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 01/42] pipeline: add new SWX pipeline type 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu ` (41 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new improved Software Switch (SWX) pipeline type that supports dynamically-defined packet headers, meta-data, actions and pipelines. Actions and pipelines are defined through instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 10 ++- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 70 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 79 ++++++++++++++++++++ 4 files changed, 160 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d70b1a023..880c2b274 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -1,6 +1,12 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c') -headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h') +sources = files('rte_pipeline.c', + 'rte_port_in_action.c', + 'rte_table_action.c', + 'rte_swx_pipeline.c',) +headers = files('rte_pipeline.h', + 'rte_port_in_action.h', + 'rte_table_action.h', + 'rte_swx_pipeline.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 9ed80eb04..39593f1ee 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -55,4 +55,7 @@ EXPERIMENTAL { rte_table_action_time_read; rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; + rte_swx_pipeline_config; + rte_swx_pipeline_build; + rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c new file mode 100644 index 000000000..2319d4570 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define CHECK_NAME(name, err_code) \ + CHECK((name) && (name)[0], err_code) + +/* + * Pipeline. + */ +struct rte_swx_pipeline { + int build_done; + int numa_node; +}; + + +/* + * Pipeline. + */ +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) +{ + struct rte_swx_pipeline *pipeline; + + /* Check input parameters. */ + CHECK(p, EINVAL); + + /* Memory allocation. */ + pipeline = calloc(1, sizeof(struct rte_swx_pipeline)); + CHECK(pipeline, ENOMEM); + + /* Initialization. */ + pipeline->numa_node = numa_node; + + *p = pipeline; + return 0; +} + +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p) +{ + if (!p) + return; + + free(p); +} + +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p) +{ + CHECK(p, EINVAL); + CHECK(p->build_done == 0, EEXIST); + + p->build_done = 1; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h new file mode 100644 index 000000000..ded26a4e4 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PIPELINE_H__ +#define __INCLUDE_RTE_SWX_PIPELINE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +/* + * Pipeline setup and operation + */ + +/** Pipeline opaque data structure. */ +struct rte_swx_pipeline; + +/** + * Pipeline configure + * + * @param[out] p + * Pipeline handle. Must point to valid memory. Contains valid pipeline handle + * when the function returns successfully. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_config(struct rte_swx_pipeline **p, + int numa_node); + +/** + * Pipeline build + * + * Once called, the pipeline build operation marks the end of pipeline + * configuration. At this point, all the internal data structures needed to run + * the pipeline are built. + * + * @param[in] p + * Pipeline handle. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Pipeline was already built successfully. + */ +__rte_experimental +int +rte_swx_pipeline_build(struct rte_swx_pipeline *p); + +/** + * Pipeline free + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_free(struct rte_swx_pipeline *p); + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 02/42] pipeline: add SWX pipeline input port 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (40 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 5 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 03/42] pipeline: add SWX pipeline output port 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (39 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 04/42] pipeline: add SWX headers and meta-data 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (38 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 05/42] pipeline: add SWX extern objects and funcs 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu ` (37 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 5 files changed, 694 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 06/42] pipeline: add SWX pipeline action 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (36 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 07/42] pipeline: add SWX pipeline tables 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (35 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..4d2af1be6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match_type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..dc434b72e --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to facilitate the membership of this table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation searches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 08/42] pipeline: add SWX pipeline instructions 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu ` (34 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4d2af1be6..ec76294b0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 09/42] pipeline: add SWX Rx and extract instructions 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu ` (33 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add packet reception and header extraction instructions. The Rx must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ec76294b0..aae3d84b2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 10/42] pipeline: add SWX Tx and emit instructions 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (32 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. Tx ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 11/42] pipeline: add header validate and invalidate SWX instructions 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 12/42] pipeline: add SWX move instruction Cristian Dumitrescu ` (31 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 12/42] pipeline: add SWX move instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu ` (30 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 13/42] pipeline: add SWX DMA instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 12/42] pipeline: add SWX move instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (29 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 14/42] pipeline: introduce SWX add instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu ` (28 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 15/42] pipeline: introduce SWX subtract instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (27 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 16/42] pipeline: introduce SWX ckadd instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (26 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 17/42] pipeline: introduce SWX cksub instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (25 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 18/42] pipeline: introduce SWX and instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (24 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 19/42] pipeline: introduce SWX or instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu ` (23 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 20/42] pipeline: introduce SWX XOR instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu ` (22 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 21/42] pipeline: introduce SWX SHL instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu ` (21 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 22/42] pipeline: introduce SWX SHR instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (20 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 23/42] pipeline: introduce SWX table instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (19 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 24/42] pipeline: introduce SWX extern instruction 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu ` (18 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance to the mailbox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 25/42] pipeline: introduce SWX jump and return instructions 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 26/42] pipeline: add SWX instruction description Cristian Dumitrescu ` (17 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 26/42] pipeline: add SWX instruction description 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (16 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index aae3d84b2..a1534b043 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 27/42] pipeline: add SWX instruction verifier 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 26/42] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (15 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 28/42] pipeline: add SWX instruction optimizer 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (14 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 29/42] pipeline: add SWX pipeline query API 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (13 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..54b895f0a 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action_arg + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup operation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 30/42] pipeline: add SWX pipeline flush 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu ` (12 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index a1534b043..7b131b0de 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-10-01 10:19 ` Cristian Dumitrescu 2020-10-01 12:03 ` David Marchand 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (11 subsequent siblings) 42 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:19 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 4 files changed, 1736 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..576fb2bf3 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wildcard and LPM tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index 54b895f0a..bab189494 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-10-01 12:03 ` David Marchand 0 siblings, 0 replies; 329+ messages in thread From: David Marchand @ 2020-10-01 12:03 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon, Ray Kinsella On Thu, Oct 1, 2020 at 12:21 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map > index 730e11a0c..ec38f0eef 100644 > --- a/lib/librte_pipeline/rte_pipeline_version.map > +++ b/lib/librte_pipeline/rte_pipeline_version.map > @@ -1,4 +1,4 @@ > -DPDK_21 { > +DPDK_20.0 { Ouch. Fixed when applying... > global: > > rte_pipeline_ah_packet_drop; -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-04 7:50 ` Raslan Darawsheh 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 33/42] port: add ethernet device SWX port Cristian Dumitrescu ` (10 subsequent siblings) 42 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 7b131b0de..aed627393 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-10-04 7:50 ` Raslan Darawsheh 2020-10-04 9:01 ` David Marchand 2020-10-04 18:28 ` Dumitrescu, Cristian 0 siblings, 2 replies; 329+ messages in thread From: Raslan Darawsheh @ 2020-10-04 7:50 UTC (permalink / raw) To: Cristian Dumitrescu, dev; +Cc: NBU-Contact-Thomas Monjalon, david.marchand Hi, Resending to reply on the correct version, > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Cristian Dumitrescu > Sent: Wednesday, September 30, 2020 9:34 AM > To: dev@dpdk.org > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; > david.marchand@redhat.com > Subject: [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline > specification file > > Add support for building the SWX pipeline based on specification file > with syntax aligned to the P4 language. The specification file may be > generated by the P4C compiler in the future. > > Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> > --- > lib/librte_pipeline/meson.build | 1 + > lib/librte_pipeline/rte_pipeline_version.map | 1 + > lib/librte_pipeline/rte_swx_pipeline.h | 26 + > lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ > 4 files changed, 1467 insertions(+) > create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c > > diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build > index be1d9c3a4..65c1a8d6a 100644 > --- a/lib/librte_pipeline/meson.build > +++ b/lib/librte_pipeline/meson.build Seems like this patch introduced a new failure in compilation as following: ./../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration] new_fields = reallocarray(s->fields, ^~~~~~~~~~~~ realloc ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: nested extern declaration of 'reallocarray' [-Werror=nested-externs] ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:13: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_fields = reallocarray(s->fields, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'action_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:455:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_instructions = reallocarray(s->instructions, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'table_key_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:623:13: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_fields = reallocarray(s->params.fields, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'table_actions_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:703:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_action_names = reallocarray(s->params.action_names, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'apply_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:1022:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_instructions = reallocarray(s->instructions, ^ cc1: all warnings being treated as errors [502/2328] Compiling C object lib/librte_vhost.a.p/librte_vhost_vhost_crypto.c.o ninja: build stopped: subcommand failed. This is reproducible in the following environment: ARCH: aarch64 CC: gcc OS: Fedora 32 (Container Image) Kernel: 4.4.0-104-generic GCC(aarch64): aarch64-linux-gnu-gcc (Linaro GCC 7.1-2017.08) 7.1.1 20170707 Can you kindly help with providing a fix for it ? Kindest regards Raslan Darawsheh [Raslan Darawsheh] ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file 2020-10-04 7:50 ` Raslan Darawsheh @ 2020-10-04 9:01 ` David Marchand 2020-10-04 10:47 ` Raslan Darawsheh 2020-10-04 18:28 ` Dumitrescu, Cristian 1 sibling, 1 reply; 329+ messages in thread From: David Marchand @ 2020-10-04 9:01 UTC (permalink / raw) To: Raslan Darawsheh; +Cc: Cristian Dumitrescu, dev, Thomas Monjalon Hello Raslan, On Sun, Oct 4, 2020 at 9:51 AM Raslan Darawsheh <rasland@nvidia.com> wrote: > Seems like this patch introduced a new failure in compilation as following: > > ./../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration] > new_fields = reallocarray(s->fields, > ^~~~~~~~~~~~ > realloc [...] > > This is reproducible in the following environment: > ARCH: aarch64 > CC: gcc > OS: Fedora 32 (Container Image) > Kernel: 4.4.0-104-generic > GCC(aarch64): aarch64-linux-gnu-gcc (Linaro GCC 7.1-2017.08) 7.1.1 20170707 > > Can you kindly help with providing a fix for it ? https://git.dpdk.org/dpdk/commit/?id=2a8be2ff7530c5031c5b29bf4f0353e3d2d4aee0 -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file 2020-10-04 9:01 ` David Marchand @ 2020-10-04 10:47 ` Raslan Darawsheh 0 siblings, 0 replies; 329+ messages in thread From: Raslan Darawsheh @ 2020-10-04 10:47 UTC (permalink / raw) To: David Marchand; +Cc: Cristian Dumitrescu, dev, NBU-Contact-Thomas Monjalon Hi David, Seems that I missed this patch, thanks for the help and pointing me to it, appreciated. Kindest regards, Raslan Darawsheh > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Sunday, October 4, 2020 12:01 PM > To: Raslan Darawsheh <rasland@nvidia.com> > Cc: Cristian Dumitrescu <cristian.dumitrescu@intel.com>; dev@dpdk.org; > NBU-Contact-Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline > specification file > > Hello Raslan, > > On Sun, Oct 4, 2020 at 9:51 AM Raslan Darawsheh <rasland@nvidia.com> > wrote: > > Seems like this patch introduced a new failure in compilation as following: > > > > ./../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: > implicit declaration of function 'reallocarray'; did you mean 'realloc'? [- > Werror=implicit-function-declaration] > > new_fields = reallocarray(s->fields, > > ^~~~~~~~~~~~ > > realloc > > [...] > > > > > This is reproducible in the following environment: > > ARCH: aarch64 > > CC: gcc > > OS: Fedora 32 (Container Image) > > Kernel: 4.4.0-104-generic > > GCC(aarch64): aarch64-linux-gnu-gcc (Linaro GCC 7.1-2017.08) 7.1.1 > 20170707 > > > > Can you kindly help with providing a fix for it ? > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.d > pdk.org%2Fdpdk%2Fcommit%2F%3Fid%3D2a8be2ff7530c5031c5b29bf4f0353 > e3d2d4aee0&data=02%7C01%7Crasland%40nvidia.com%7C583ee2a555 > 34409aa86f08d868441f5e%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0 > %7C637373989082245301&sdata=8a5dF6yoVfvvAwUVcnptUvhUSCDb% > 2BXsfPc2e5NXN15A%3D&reserved=0 > > > -- > David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file 2020-10-04 7:50 ` Raslan Darawsheh 2020-10-04 9:01 ` David Marchand @ 2020-10-04 18:28 ` Dumitrescu, Cristian 1 sibling, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-10-04 18:28 UTC (permalink / raw) To: Raslan Darawsheh, dev; +Cc: NBU-Contact-Thomas Monjalon, david.marchand > -----Original Message----- > From: Raslan Darawsheh <rasland@nvidia.com> > Sent: Sunday, October 4, 2020 8:51 AM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; dev@dpdk.org > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; > david.marchand@redhat.com > Subject: RE: [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline > specification file > > Hi, > > Resending to reply on the correct version, > > > -----Original Message----- > > From: dev <dev-bounces@dpdk.org> On Behalf Of Cristian Dumitrescu > > Sent: Wednesday, September 30, 2020 9:34 AM > > To: dev@dpdk.org > > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; > > david.marchand@redhat.com > > Subject: [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline > > specification file > > > > Add support for building the SWX pipeline based on specification file > > with syntax aligned to the P4 language. The specification file may be > > generated by the P4C compiler in the future. > > > > Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> > > --- > > lib/librte_pipeline/meson.build | 1 + > > lib/librte_pipeline/rte_pipeline_version.map | 1 + > > lib/librte_pipeline/rte_swx_pipeline.h | 26 + > > lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 > ++++++++++++++++++ > > 4 files changed, 1467 insertions(+) > > create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c > > > > diff --git a/lib/librte_pipeline/meson.build > b/lib/librte_pipeline/meson.build > > index be1d9c3a4..65c1a8d6a 100644 > > --- a/lib/librte_pipeline/meson.build > > +++ b/lib/librte_pipeline/meson.build > > Seems like this patch introduced a new failure in compilation as following: > > ./../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: > implicit declaration of function 'reallocarray'; did you mean 'realloc'? [- > Werror=implicit-function-declaration] > new_fields = reallocarray(s->fields, > ^~~~~~~~~~~~ > realloc > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: > nested extern declaration of 'reallocarray' [-Werror=nested-externs] > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:13: error: > assignment makes pointer from integer without a cast [-Werror=int- > conversion] > new_fields = reallocarray(s->fields, > ^ > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function > 'action_block_parse': > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:455:19: error: > assignment makes pointer from integer without a cast [-Werror=int- > conversion] > new_instructions = reallocarray(s->instructions, > ^ > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function > 'table_key_block_parse': > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:623:13: error: > assignment makes pointer from integer without a cast [-Werror=int- > conversion] > new_fields = reallocarray(s->params.fields, > ^ > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function > 'table_actions_block_parse': > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:703:19: error: > assignment makes pointer from integer without a cast [-Werror=int- > conversion] > new_action_names = reallocarray(s->params.action_names, > ^ > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function > 'apply_block_parse': > ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:1022:19: error: > assignment makes pointer from integer without a cast [-Werror=int- > conversion] > new_instructions = reallocarray(s->instructions, > ^ > cc1: all warnings being treated as errors > [502/2328] Compiling C object > lib/librte_vhost.a.p/librte_vhost_vhost_crypto.c.o > ninja: build stopped: subcommand failed. > > This is reproducible in the following environment: > ARCH: aarch64 > CC: gcc > OS: Fedora 32 (Container Image) > Kernel: 4.4.0-104-generic > GCC(aarch64): aarch64-linux-gnu-gcc (Linaro GCC 7.1-2017.08) 7.1.1 > 20170707 > > Can you kindly help with providing a fix for it ? > > Kindest regards > Raslan Darawsheh > [Raslan Darawsheh] Hi Raslan, This issue was already fixed by David's patch [1] which was already applied last Fri. Please update to the latest code and let us know if any issues. This issue is taking place when glibc version is older than 2.26, i.e. older than 2017. Regards, Cristian [1] http://patches.dpdk.org/patch/79525/ ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 33/42] port: add ethernet device SWX port 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 34/42] port: add source and sink SWX ports Cristian Dumitrescu ` (9 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 34/42] port: add source and sink SWX ports 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 33/42] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 35/42] table: add exact match SWX table Cristian Dumitrescu ` (8 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 35/42] table: add exact match SWX table 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 34/42] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 36/42] examples/pipeline: add new example application Cristian Dumitrescu ` (7 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 4 files changed, 892 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 36/42] examples/pipeline: add new example application 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 35/42] table: add exact match SWX table Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (6 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- MAINTAINERS | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 50 ++++ examples/pipeline/main.c | 50 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1296 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/MAINTAINERS b/MAINTAINERS index 49a6dfa7a..df3033cdb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1331,6 +1331,7 @@ F: app/test/test_table* F: app/test-pipeline/ F: doc/guides/sample_app_ug/test_pipeline.rst F: examples/ip_pipeline/ +F: examples/pipeline/ F: doc/guides/sample_app_ug/ip_pipeline.rst diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..da2f4850b --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. -DALLOW_EXPERIMENTAL_API -D_GNU_SOURCE + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..d831df15e --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + return 0; +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..1ebef3a9d --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..84bbcf2b2 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..e6351fd27 --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..7ff22e97f --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Control thread: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Control thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Control thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Control thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Control thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..d9d8645d4 --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 37/42] examples/pipeline: add message passing mechanism 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 36/42] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu ` (5 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 137 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 519 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index da2f4850b..097847b37 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..e168c4dda --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..f0a73d87c --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index d831df15e..3573a77f5 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,5 +167,19 @@ main(int argc, char **argv) NULL, SKIP_MASTER); - return 0; + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; + + /* Dispatch loop */ + for ( ; ; ) { + conn_poll_for_conn(conn); + + conn_poll_for_msg(conn); + } } diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index 1ebef3a9d..b92851049 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 38/42] examples/pipeline: add configuration commands 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (4 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 10 +- examples/pipeline/meson.build | 1 + 5 files changed, 1430 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 097847b37..d0a1f02e1 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..9f3d87a3c --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %s\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + rte_eth_link_speed_to_str(eth_link.link_speed), + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..a23a42686 --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 3573a77f5..2303c9e46 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index b92851049..e47d483de 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 39/42] examples/pipeline: add l2fwd example 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu ` (3 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..d89caf2d0 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..0cf5e32fa --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 40/42] examples/pipeline: add l2fwd with MAC swap example 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu ` (2 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..0f2a89ac5 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..043379cdd --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 41/42] examples/pipeline: add VXLAN encapsulation example 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu 2020-10-01 17:15 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language David Marchand 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..7bf4a5757 --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/pipeline/examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/pipeline/examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..b3f28630d --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to headers and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..1636ba080 --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v7 42/42] doc: add new SWX pipeline type to release notes 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (40 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu @ 2020-10-01 10:20 ` Cristian Dumitrescu 2020-10-01 17:15 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language David Marchand 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-10-01 10:20 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the new SWX pipeline type to the release notes. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- doc/guides/rel_notes/release_20_11.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index 4eb3224a7..15ec247d7 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -78,6 +78,17 @@ New Features ``--portmask=N`` where N represents the hexadecimal bitmask of ports used. +* **Updated the pipeline library for alignment with the P4 language.** + + Added new Software Switch (SWX) pipeline type that provides more + flexibility through API and feature alignment with the P4 language. + + * The packet headers, meta-data, actions, tables and pipelines are + dynamically defined instead of selected from pre-defined set. + * The actions and the pipeline are defined with instructions. + * Extern objects and functions can be plugged into the pipeline. + * Transaction-oriented table updates. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (41 preceding siblings ...) 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu @ 2020-10-01 17:15 ` David Marchand 2020-10-01 17:22 ` Dumitrescu, Cristian 42 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-10-01 17:15 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon On Thu, Oct 1, 2020 at 12:20 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > This patch set introduces a new pipeline type that combines the DPDK > performance with the flexibility of the P4-16 language[1]. The new API > can be used either by itself to code a complete software switch (SWX) > or data plane app, or in combination with the open-source P4 compiler > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > programs to be translated to the DPDK API and run on multi-core CPUs. > > Main new features: > > * Nothing is hard-wired, everything is dynamically defined: The packet > headers (i.e. protocols), the packet meta-data, the actions, the > tables and the pipeline itself are dynamically defined instead of > having to be selected from a pre-defined set. > > * Instructions: The actions and the life of the packet through the > pipeline are defined with instructions that manipulate the pipeline > objects mentioned above. The pipeline is the main function of the > packet program, with actions as subroutines triggered by the tables. > > * Call external plugins: Extern objects and functions can be defined > to call functionality that cannot be efficiently implemented with > the existing pipeline-oriented instruction set, such as: special > error detecting/correcting codes, crypto, meters, stats arrays, > heuristics, etc. > > * Better control plane interaction: Transaction-oriented table update > mechanism that supports multi-table atomic updates. Multiple tables > can be updated in a single step with only the before and after table > sets visible to the packets. Alignment with P4Runtime [3]. > > * Performance: Multiple packets are in-flight within the pipeline at > any moment. Each packet is owned by a different time-sharing thread > in run-to-completion, with the thread pausing before memory access > operations such as packet I/O and table lookup to allow the memory > prefetch to complete. The instructions are verified and translated > at initialization time with no run-time impact. The instructions are > also optimized to detect and "fuse" frequently used patterns into > vector-like instructions transparently to the user. > > API deprecation and maturing roadmap: > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > prior to and removed as part of the DPDK 21.11 LTS release. > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > and become stable as part of the same DPDK 21.11 LTS release. > > V7 changes: > * Set copyright year to 2020 for all new files. > * Fixed two Doxygen warnings. > > V6 changes: > * Fixed issues in the example app Makefile. > * Used rte_eth_link_speed_to_str() in the example app. > * Added release notes update. > > V5 changes: > * Upper case abberviations in some commit titles. > * Added new example app in the MAINTAINERS file. > * Absolutely no code changes. > > V4 changes: > * Spell check fixes. > > V3 changes: > * Removed the library Makefile support to align with the latest DPDK. > > V2 changes: > * Updated the title and commit messages to reflect the introduction of > the new SWX pipeline type. > * Added the API deprecation and maturing roadmap to the cover letter. > * Added support for building the SWX pipeline based on specification > file with syntax aligned to the P4 language. The spec file may be > generated by the P4C compiler in the future (see patch 32). Reworked > the examples accordingly (see patches 39, 40 and 41). > * Added support for the SWX sink port (used for packet drop or log) > when PCAP library is disabled from the build. > * Added checks to the application CLI commands to prevent execution > when dependencies of the current command have previously failed (see > patch 38). > * Fixed build warning for 32-bit targets due to the printing of 64-bit > statistics counters (see patch 38). > > [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf > [2] P4-16 compiler: https://github.com/p4lang/p4c > [3] P4Runtime specification: > https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf > > Cristian Dumitrescu (41): > pipeline: add new SWX pipeline type > pipeline: add SWX pipeline input port > pipeline: add SWX pipeline output port > pipeline: add SWX headers and meta-data > pipeline: add SWX extern objects and funcs > pipeline: add SWX pipeline action > pipeline: add SWX pipeline tables > pipeline: add SWX pipeline instructions > pipeline: add SWX Rx and extract instructions > pipeline: add SWX Tx and emit instructions > pipeline: add header validate and invalidate SWX instructions > pipeline: add SWX move instruction > pipeline: add SWX DMA instruction > pipeline: introduce SWX add instruction > pipeline: introduce SWX subtract instruction > pipeline: introduce SWX ckadd instruction > pipeline: introduce SWX cksub instruction > pipeline: introduce SWX and instruction > pipeline: introduce SWX or instruction > pipeline: introduce SWX XOR instruction > pipeline: introduce SWX SHL instruction > pipeline: introduce SWX SHR instruction > pipeline: introduce SWX table instruction > pipeline: introduce SWX extern instruction > pipeline: introduce SWX jump and return instructions > pipeline: add SWX instruction description > pipeline: add SWX instruction verifier > pipeline: add SWX instruction optimizer > pipeline: add SWX pipeline query API > pipeline: add SWX pipeline flush > pipeline: add SWX table update high level API > pipeline: add SWX pipeline specification file > port: add ethernet device SWX port > port: add source and sink SWX ports > table: add exact match SWX table > examples/pipeline: add new example application > examples/pipeline: add message passing mechanism > examples/pipeline: add configuration commands > examples/pipeline: add l2fwd example > examples/pipeline: add l2fwd with MAC swap example > examples/pipeline: add VXLAN encapsulation example > doc: add new SWX pipeline type to release notes > - Fixed pipeline .map file - Annotated/sorted the experimental symbols in .map files touched by this series, - Updated doxygen index to include the new headers, - Switched the vxlan python script (coming with the pipeline example) to python3 + fixed some pylint trivial warnings, Series applied. -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language 2020-10-01 17:15 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language David Marchand @ 2020-10-01 17:22 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-10-01 17:22 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Thursday, October 1, 2020 6:16 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [PATCH v7 00/42] Pipeline alignment with the P4 language > > On Thu, Oct 1, 2020 at 12:20 PM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > > > > This patch set introduces a new pipeline type that combines the DPDK > > performance with the flexibility of the P4-16 language[1]. The new API > > can be used either by itself to code a complete software switch (SWX) > > or data plane app, or in combination with the open-source P4 compiler > > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > > programs to be translated to the DPDK API and run on multi-core CPUs. > > > > Main new features: > > > > * Nothing is hard-wired, everything is dynamically defined: The packet > > headers (i.e. protocols), the packet meta-data, the actions, the > > tables and the pipeline itself are dynamically defined instead of > > having to be selected from a pre-defined set. > > > > * Instructions: The actions and the life of the packet through the > > pipeline are defined with instructions that manipulate the pipeline > > objects mentioned above. The pipeline is the main function of the > > packet program, with actions as subroutines triggered by the tables. > > > > * Call external plugins: Extern objects and functions can be defined > > to call functionality that cannot be efficiently implemented with > > the existing pipeline-oriented instruction set, such as: special > > error detecting/correcting codes, crypto, meters, stats arrays, > > heuristics, etc. > > > > * Better control plane interaction: Transaction-oriented table update > > mechanism that supports multi-table atomic updates. Multiple tables > > can be updated in a single step with only the before and after table > > sets visible to the packets. Alignment with P4Runtime [3]. > > > > * Performance: Multiple packets are in-flight within the pipeline at > > any moment. Each packet is owned by a different time-sharing thread > > in run-to-completion, with the thread pausing before memory access > > operations such as packet I/O and table lookup to allow the memory > > prefetch to complete. The instructions are verified and translated > > at initialization time with no run-time impact. The instructions are > > also optimized to detect and "fuse" frequently used patterns into > > vector-like instructions transparently to the user. > > > > API deprecation and maturing roadmap: > > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > > prior to and removed as part of the DPDK 21.11 LTS release. > > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > > and become stable as part of the same DPDK 21.11 LTS release. > > > > V7 changes: > > * Set copyright year to 2020 for all new files. > > * Fixed two Doxygen warnings. > > > > V6 changes: > > * Fixed issues in the example app Makefile. > > * Used rte_eth_link_speed_to_str() in the example app. > > * Added release notes update. > > > > V5 changes: > > * Upper case abberviations in some commit titles. > > * Added new example app in the MAINTAINERS file. > > * Absolutely no code changes. > > > > V4 changes: > > * Spell check fixes. > > > > V3 changes: > > * Removed the library Makefile support to align with the latest DPDK. > > > > V2 changes: > > * Updated the title and commit messages to reflect the introduction of > > the new SWX pipeline type. > > * Added the API deprecation and maturing roadmap to the cover letter. > > * Added support for building the SWX pipeline based on specification > > file with syntax aligned to the P4 language. The spec file may be > > generated by the P4C compiler in the future (see patch 32). Reworked > > the examples accordingly (see patches 39, 40 and 41). > > * Added support for the SWX sink port (used for packet drop or log) > > when PCAP library is disabled from the build. > > * Added checks to the application CLI commands to prevent execution > > when dependencies of the current command have previously failed (see > > patch 38). > > * Fixed build warning for 32-bit targets due to the printing of 64-bit > > statistics counters (see patch 38). > > > > [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf > > [2] P4-16 compiler: https://github.com/p4lang/p4c > > [3] P4Runtime specification: > > https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf > > > > Cristian Dumitrescu (41): > > pipeline: add new SWX pipeline type > > pipeline: add SWX pipeline input port > > pipeline: add SWX pipeline output port > > pipeline: add SWX headers and meta-data > > pipeline: add SWX extern objects and funcs > > pipeline: add SWX pipeline action > > pipeline: add SWX pipeline tables > > pipeline: add SWX pipeline instructions > > pipeline: add SWX Rx and extract instructions > > pipeline: add SWX Tx and emit instructions > > pipeline: add header validate and invalidate SWX instructions > > pipeline: add SWX move instruction > > pipeline: add SWX DMA instruction > > pipeline: introduce SWX add instruction > > pipeline: introduce SWX subtract instruction > > pipeline: introduce SWX ckadd instruction > > pipeline: introduce SWX cksub instruction > > pipeline: introduce SWX and instruction > > pipeline: introduce SWX or instruction > > pipeline: introduce SWX XOR instruction > > pipeline: introduce SWX SHL instruction > > pipeline: introduce SWX SHR instruction > > pipeline: introduce SWX table instruction > > pipeline: introduce SWX extern instruction > > pipeline: introduce SWX jump and return instructions > > pipeline: add SWX instruction description > > pipeline: add SWX instruction verifier > > pipeline: add SWX instruction optimizer > > pipeline: add SWX pipeline query API > > pipeline: add SWX pipeline flush > > pipeline: add SWX table update high level API > > pipeline: add SWX pipeline specification file > > port: add ethernet device SWX port > > port: add source and sink SWX ports > > table: add exact match SWX table > > examples/pipeline: add new example application > > examples/pipeline: add message passing mechanism > > examples/pipeline: add configuration commands > > examples/pipeline: add l2fwd example > > examples/pipeline: add l2fwd with MAC swap example > > examples/pipeline: add VXLAN encapsulation example > > doc: add new SWX pipeline type to release notes > > > > - Fixed pipeline .map file > - Annotated/sorted the experimental symbols in .map files touched by > this series, > - Updated doxygen index to include the new headers, > - Switched the vxlan python script (coming with the pipeline example) > to python3 + fixed some pylint trivial warnings, > > Series applied. > > > -- > David Marchand Thanks for your support, David! ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 02/42] pipeline: add SWX pipeline input port 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (40 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 5 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 03/42] pipeline: add SWX pipeline output port 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (39 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 04/42] pipeline: add SWX headers and meta-data 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (38 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 05/42] pipeline: add SWX extern objects and funcs 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu ` (37 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 5 files changed, 694 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 06/42] pipeline: add SWX pipeline action 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (36 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 07/42] pipeline: add SWX pipeline tables 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (35 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..dc434b72e --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to facilitate the membership of this table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation searches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 08/42] pipeline: add SWX pipeline instructions 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu ` (34 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 09/42] pipeline: add SWX Rx and extract instructions 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu ` (33 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add packet reception and header extraction instructions. The Rx must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 10/42] pipeline: add SWX Tx and emit instructions 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (32 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. Tx ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 11/42] pipeline: add header validate and invalidate SWX instructions 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 12/42] pipeline: add SWX move instruction Cristian Dumitrescu ` (31 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 12/42] pipeline: add SWX move instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu ` (30 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 13/42] pipeline: add SWX DMA instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 12/42] pipeline: add SWX move instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (29 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 14/42] pipeline: introduce SWX add instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu ` (28 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 15/42] pipeline: introduce SWX subtract instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (27 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 16/42] pipeline: introduce SWX ckadd instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (26 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 17/42] pipeline: introduce SWX cksub instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (25 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 18/42] pipeline: introduce SWX and instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (24 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 19/42] pipeline: introduce SWX or instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu ` (23 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 20/42] pipeline: introduce SWX XOR instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu ` (22 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 21/42] pipeline: introduce SWX SHL instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu ` (21 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 22/42] pipeline: introduce SWX SHR instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (20 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 23/42] pipeline: introduce SWX table instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (19 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 24/42] pipeline: introduce SWX extern instruction 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu ` (18 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance to the mailbox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 25/42] pipeline: introduce SWX jump and return instructions 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-09-30 6:33 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 26/42] pipeline: add SWX instruction description Cristian Dumitrescu ` (17 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:33 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 26/42] pipeline: add SWX instruction description 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (16 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..d6c086e27 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 27/42] pipeline: add SWX instruction verifier 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 26/42] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (15 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 28/42] pipeline: add SWX instruction optimizer 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (14 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 29/42] pipeline: add SWX pipeline query API 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (13 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..344c7c833 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup operation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 30/42] pipeline: add SWX pipeline flush 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu ` (12 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d6c086e27..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 31/42] pipeline: add SWX table update high level API 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (11 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 4 files changed, 1736 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..576fb2bf3 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wildcard and LPM tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index 344c7c833..18b065834 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-10-04 7:48 ` Raslan Darawsheh 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 33/42] port: add ethernet device SWX port Cristian Dumitrescu ` (10 subsequent siblings) 42 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 6da5710af..6928e78b6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-10-04 7:48 ` Raslan Darawsheh 0 siblings, 0 replies; 329+ messages in thread From: Raslan Darawsheh @ 2020-10-04 7:48 UTC (permalink / raw) To: Cristian Dumitrescu, dev; +Cc: NBU-Contact-Thomas Monjalon, david.marchand Hi, > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Cristian Dumitrescu > Sent: Wednesday, September 30, 2020 9:34 AM > To: dev@dpdk.org > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; > david.marchand@redhat.com > Subject: [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline > specification file > > Add support for building the SWX pipeline based on specification file > with syntax aligned to the P4 language. The specification file may be > generated by the P4C compiler in the future. > > Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> > --- > lib/librte_pipeline/meson.build | 1 + > lib/librte_pipeline/rte_pipeline_version.map | 1 + > lib/librte_pipeline/rte_swx_pipeline.h | 26 + > lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ > 4 files changed, 1467 insertions(+) > create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c > > diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build > index be1d9c3a4..65c1a8d6a 100644 > --- a/lib/librte_pipeline/meson.build > +++ b/lib/librte_pipeline/meson.build Seems like this patch introduced a new failure in compilation as following: ./../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration] new_fields = reallocarray(s->fields, ^~~~~~~~~~~~ realloc ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:15: error: nested extern declaration of 'reallocarray' [-Werror=nested-externs] ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:216:13: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_fields = reallocarray(s->fields, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'action_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:455:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_instructions = reallocarray(s->instructions, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'table_key_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:623:13: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_fields = reallocarray(s->params.fields, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'table_actions_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:703:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_action_names = reallocarray(s->params.action_names, ^ ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c: In function 'apply_block_parse': ../../root/dpdk/lib/librte_pipeline/rte_swx_pipeline_spec.c:1022:19: error: assignment makes pointer from integer without a cast [-Werror=int-conversion] new_instructions = reallocarray(s->instructions, ^ cc1: all warnings being treated as errors [502/2328] Compiling C object lib/librte_vhost.a.p/librte_vhost_vhost_crypto.c.o ninja: build stopped: subcommand failed. This is reproducible in the following environment: ARCH: aarch64 CC: gcc OS: Fedora 32 (Container Image) Kernel: 4.4.0-104-generic GCC(aarch64): aarch64-linux-gnu-gcc (Linaro GCC 7.1-2017.08) 7.1.1 20170707 Can you kindly help with providing a fix for it ? Kindest regards Raslan Darawsheh ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 33/42] port: add ethernet device SWX port 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 34/42] port: add source and sink SWX ports Cristian Dumitrescu ` (9 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 34/42] port: add source and sink SWX ports 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 33/42] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 35/42] table: add exact match SWX table Cristian Dumitrescu ` (8 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 35/42] table: add exact match SWX table 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 34/42] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 36/42] examples/pipeline: add new example application Cristian Dumitrescu ` (7 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 4 files changed, 892 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 36/42] examples/pipeline: add new example application 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 35/42] table: add exact match SWX table Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (6 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- MAINTAINERS | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 50 ++++ examples/pipeline/main.c | 50 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1296 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/MAINTAINERS b/MAINTAINERS index 49a6dfa7a..df3033cdb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1331,6 +1331,7 @@ F: app/test/test_table* F: app/test-pipeline/ F: doc/guides/sample_app_ug/test_pipeline.rst F: examples/ip_pipeline/ +F: examples/pipeline/ F: doc/guides/sample_app_ug/ip_pipeline.rst diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..da2f4850b --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. -DALLOW_EXPERIMENTAL_API -D_GNU_SOURCE + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..dec78fba5 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + return 0; +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..688870f97 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..1be9828f0 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Control thread: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Control thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Control thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Control thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Control thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 37/42] examples/pipeline: add message passing mechanism 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 36/42] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu ` (5 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 137 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 519 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index da2f4850b..097847b37 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dec78fba5..dc5a72899 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,5 +167,19 @@ main(int argc, char **argv) NULL, SKIP_MASTER); - return 0; + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; + + /* Dispatch loop */ + for ( ; ; ) { + conn_poll_for_conn(conn); + + conn_poll_for_msg(conn); + } } diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 38/42] examples/pipeline: add configuration commands 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (4 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 10 +- examples/pipeline/meson.build | 1 + 5 files changed, 1430 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 097847b37..d0a1f02e1 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..6fb5a891c --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %s\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + rte_eth_link_speed_to_str(eth_link.link_speed), + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dc5a72899..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 39/42] examples/pipeline: add l2fwd example 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu ` (3 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..c6a3b9d04 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..be7773b58 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 40/42] examples/pipeline: add l2fwd with MAC swap example 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu ` (2 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..8031b2655 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..9044d7d7f --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 41/42] examples/pipeline: add VXLAN encapsulation example 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu 2020-09-30 19:34 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language David Marchand 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..f1efd177e --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/pipeline/examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/pipeline/examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..aaa105da7 --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to headers and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c6975343e --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v6 42/42] doc: add new SWX pipeline type to release notes 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (40 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu @ 2020-09-30 6:34 ` Cristian Dumitrescu 2020-09-30 19:34 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language David Marchand 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-30 6:34 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the new SWX pipeline type to the release notes. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- doc/guides/rel_notes/release_20_11.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index 4eb3224a7..15ec247d7 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -78,6 +78,17 @@ New Features ``--portmask=N`` where N represents the hexadecimal bitmask of ports used. +* **Updated the pipeline library for alignment with the P4 language.** + + Added new Software Switch (SWX) pipeline type that provides more + flexibility through API and feature alignment with the P4 language. + + * The packet headers, meta-data, actions, tables and pipelines are + dynamically defined instead of selected from pre-defined set. + * The actions and the pipeline are defined with instructions. + * Extern objects and functions can be plugged into the pipeline. + * Transaction-oriented table updates. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu ` (41 preceding siblings ...) 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu @ 2020-09-30 19:34 ` David Marchand 2020-10-01 10:45 ` Dumitrescu, Cristian 42 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-30 19:34 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon On Wed, Sep 30, 2020 at 8:34 AM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > This patch set introduces a new pipeline type that combines the DPDK > performance with the flexibility of the P4-16 language[1]. The new API > can be used either by itself to code a complete software switch (SWX) > or data plane app, or in combination with the open-source P4 compiler > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > programs to be translated to the DPDK API and run on multi-core CPUs. > > Main new features: > > * Nothing is hard-wired, everything is dynamically defined: The packet > headers (i.e. protocols), the packet meta-data, the actions, the > tables and the pipeline itself are dynamically defined instead of > having to be selected from a pre-defined set. > > * Instructions: The actions and the life of the packet through the > pipeline are defined with instructions that manipulate the pipeline > objects mentioned above. The pipeline is the main function of the > packet program, with actions as subroutines triggered by the tables. > > * Call external plugins: Extern objects and functions can be defined > to call functionality that cannot be efficiently implemented with > the existing pipeline-oriented instruction set, such as: special > error detecting/correcting codes, crypto, meters, stats arrays, > heuristics, etc. > > * Better control plane interaction: Transaction-oriented table update > mechanism that supports multi-table atomic updates. Multiple tables > can be updated in a single step with only the before and after table > sets visible to the packets. Alignment with P4Runtime [3]. > > * Performance: Multiple packets are in-flight within the pipeline at > any moment. Each packet is owned by a different time-sharing thread > in run-to-completion, with the thread pausing before memory access > operations such as packet I/O and table lookup to allow the memory > prefetch to complete. The instructions are verified and translated > at initialization time with no run-time impact. The instructions are > also optimized to detect and "fuse" frequently used patterns into > vector-like instructions transparently to the user. > > API deprecation and maturing roadmap: > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > prior to and removed as part of the DPDK 21.11 LTS release. > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > and become stable as part of the same DPDK 21.11 LTS release. > > V6 changes: > * Fixed issues in the example app Makefile. > * Used rte_eth_link_speed_to_str() in the example app. > * Added release notes update. - My comment on the Copyright was meant for the whole example code, not only the makefile. - The documentation generation shows following warnings: $ ninja -v -C build doc [...] /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_ctl.h:165: warning: argument 'action' of command @param is not found in the argument list of rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, uint32_t action_id, uint32_t action_arg_id, struct rte_swx_ctl_action_arg_info *action_arg) /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_ctl.h:182: warning: The following parameters of rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, uint32_t action_id, uint32_t action_arg_id, struct rte_swx_ctl_action_arg_info *action_arg) are not documented: parameter 'action_arg' /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_pipeline.h:494: warning: argument 'match' of command @param is not found in the argument list of rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, const char *name, enum rte_swx_table_match_type match_type, struct rte_swx_table_ops *ops) /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_pipeline.h:513: warning: The following parameters of rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, const char *name, enum rte_swx_table_match_type match_type, struct rte_swx_table_ops *ops) are not documented: parameter 'match_type' [...] -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language 2020-09-30 19:34 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language David Marchand @ 2020-10-01 10:45 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-10-01 10:45 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon Hi David, > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Wednesday, September 30, 2020 8:34 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [PATCH v6 00/42] Pipeline alignment with the P4 language > > On Wed, Sep 30, 2020 at 8:34 AM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > > > > This patch set introduces a new pipeline type that combines the DPDK > > performance with the flexibility of the P4-16 language[1]. The new API > > can be used either by itself to code a complete software switch (SWX) > > or data plane app, or in combination with the open-source P4 compiler > > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > > programs to be translated to the DPDK API and run on multi-core CPUs. > > > > Main new features: > > > > * Nothing is hard-wired, everything is dynamically defined: The packet > > headers (i.e. protocols), the packet meta-data, the actions, the > > tables and the pipeline itself are dynamically defined instead of > > having to be selected from a pre-defined set. > > > > * Instructions: The actions and the life of the packet through the > > pipeline are defined with instructions that manipulate the pipeline > > objects mentioned above. The pipeline is the main function of the > > packet program, with actions as subroutines triggered by the tables. > > > > * Call external plugins: Extern objects and functions can be defined > > to call functionality that cannot be efficiently implemented with > > the existing pipeline-oriented instruction set, such as: special > > error detecting/correcting codes, crypto, meters, stats arrays, > > heuristics, etc. > > > > * Better control plane interaction: Transaction-oriented table update > > mechanism that supports multi-table atomic updates. Multiple tables > > can be updated in a single step with only the before and after table > > sets visible to the packets. Alignment with P4Runtime [3]. > > > > * Performance: Multiple packets are in-flight within the pipeline at > > any moment. Each packet is owned by a different time-sharing thread > > in run-to-completion, with the thread pausing before memory access > > operations such as packet I/O and table lookup to allow the memory > > prefetch to complete. The instructions are verified and translated > > at initialization time with no run-time impact. The instructions are > > also optimized to detect and "fuse" frequently used patterns into > > vector-like instructions transparently to the user. > > > > API deprecation and maturing roadmap: > > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > > prior to and removed as part of the DPDK 21.11 LTS release. > > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > > and become stable as part of the same DPDK 21.11 LTS release. > > > > V6 changes: > > * Fixed issues in the example app Makefile. > > * Used rte_eth_link_speed_to_str() in the example app. > > * Added release notes update. > > - My comment on the Copyright was meant for the whole example code, > not only the makefile. > Yes, sorry, I made sure all the new files have the copyright year set to 2020 in the V7 just sent. > - The documentation generation shows following warnings: > > $ ninja -v -C build doc > > [...] > > /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_ctl.h:165: warning: > argument 'action' of command @param is not found in the argument list > of rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, > uint32_t action_id, uint32_t action_arg_id, struct > rte_swx_ctl_action_arg_info *action_arg) > /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_ctl.h:182: warning: > The following parameters of rte_swx_ctl_action_arg_info_get(struct > rte_swx_pipeline *p, uint32_t action_id, uint32_t action_arg_id, > struct rte_swx_ctl_action_arg_info *action_arg) are not documented: > parameter 'action_arg' > /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_pipeline.h:494: > warning: argument 'match' of command @param is not found in the > argument list of rte_swx_pipeline_table_type_register(struct > rte_swx_pipeline *p, const char *name, enum rte_swx_table_match_type > match_type, struct rte_swx_table_ops *ops) > /home/dmarchan/dpdk/lib/librte_pipeline/rte_swx_pipeline.h:513: > warning: The following parameters of > rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, const > char *name, enum rte_swx_table_match_type match_type, struct > rte_swx_table_ops *ops) are not documented: > parameter 'match_type' > > [...] > Sorry, my bad again, fixed these two Doxygen wornings in the V7 just sent. > > -- > David Marchand Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 02/41] pipeline: add SWX pipeline input port 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (39 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 5 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 03/41] pipeline: add SWX pipeline output port 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (38 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 04/41] pipeline: add SWX headers and meta-data 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (37 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 05/41] pipeline: add SWX extern objects and funcs 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu ` (36 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 5 files changed, 694 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 06/41] pipeline: add SWX pipeline action 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (35 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 07/41] pipeline: add SWX pipeline tables 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (34 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..dc434b72e --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to facilitate the membership of this table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation searches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 08/41] pipeline: add SWX pipeline instructions 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 09/41] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu ` (33 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 09/41] pipeline: add SWX Rx and extract instructions 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 10/41] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu ` (32 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add packet reception and header extraction instructions. The Rx must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 10/41] pipeline: add SWX Tx and emit instructions 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 09/41] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (31 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. Tx ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 11/41] pipeline: add header validate and invalidate SWX instructions 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 10/41] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 12/41] pipeline: add SWX move instruction Cristian Dumitrescu ` (30 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 12/41] pipeline: add SWX move instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 13/41] pipeline: add SWX DMA instruction Cristian Dumitrescu ` (29 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 13/41] pipeline: add SWX DMA instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 12/41] pipeline: add SWX move instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (28 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 14/41] pipeline: introduce SWX add instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 13/41] pipeline: add SWX DMA instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 15/41] pipeline: introduce SWX subtract instruction Cristian Dumitrescu ` (27 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 15/41] pipeline: introduce SWX subtract instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (26 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 16/41] pipeline: introduce SWX ckadd instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 15/41] pipeline: introduce SWX subtract instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (25 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 17/41] pipeline: introduce SWX cksub instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (24 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 18/41] pipeline: introduce SWX and instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (23 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 19/41] pipeline: introduce SWX or instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 20/41] pipeline: introduce SWX XOR instruction Cristian Dumitrescu ` (22 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 20/41] pipeline: introduce SWX XOR instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 21/41] pipeline: introduce SWX SHL instruction Cristian Dumitrescu ` (21 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 21/41] pipeline: introduce SWX SHL instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 20/41] pipeline: introduce SWX XOR instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 22/41] pipeline: introduce SWX SHR instruction Cristian Dumitrescu ` (20 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 22/41] pipeline: introduce SWX SHR instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 21/41] pipeline: introduce SWX SHL instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (19 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 23/41] pipeline: introduce SWX table instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 22/41] pipeline: introduce SWX SHR instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (18 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 24/41] pipeline: introduce SWX extern instruction 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 25/41] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu ` (17 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance to the mailbox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 25/41] pipeline: introduce SWX jump and return instructions 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 26/41] pipeline: add SWX instruction description Cristian Dumitrescu ` (16 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 26/41] pipeline: add SWX instruction description 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 25/41] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (15 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..d6c086e27 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 27/41] pipeline: add SWX instruction verifier 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 26/41] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (14 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 28/41] pipeline: add SWX instruction optimizer 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (13 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 29/41] pipeline: add SWX pipeline query API 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (12 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..344c7c833 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup operation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 30/41] pipeline: add SWX pipeline flush 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu ` (11 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d6c086e27..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 31/41] pipeline: add SWX table update high level API 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (10 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 4 files changed, 1736 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..576fb2bf3 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wildcard and LPM tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index 344c7c833..18b065834 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 32/41] pipeline: add SWX pipeline specification file 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 33/41] port: add ethernet device SWX port Cristian Dumitrescu ` (9 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 6da5710af..6928e78b6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 33/41] port: add ethernet device SWX port 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 34/41] port: add source and sink SWX ports Cristian Dumitrescu ` (8 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 34/41] port: add source and sink SWX ports 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 33/41] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 35/41] table: add exact match SWX table Cristian Dumitrescu ` (7 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 35/41] table: add exact match SWX table 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 34/41] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application Cristian Dumitrescu ` (6 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 4 files changed, 892 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 35/41] table: add exact match SWX table Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:26 ` Stephen Hemminger 2020-09-29 13:51 ` David Marchand 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (5 subsequent siblings) 41 siblings, 2 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- MAINTAINERS | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 51 ++++ examples/pipeline/main.c | 50 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1297 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/MAINTAINERS b/MAINTAINERS index 3b16d7a4b..ba8d55c22 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1332,6 +1332,7 @@ F: app/test/test_table* F: app/test-pipeline/ F: doc/guides/sample_app_ug/test_pipeline.rst F: examples/ip_pipeline/ +F: examples/pipeline/ F: doc/guides/sample_app_ug/ip_pipeline.rst diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..8d01fbfed --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..dec78fba5 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + return 0; +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..688870f97 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..1be9828f0 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Control thread: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Control thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Control thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Control thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Control thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-23 18:26 ` Stephen Hemminger 2020-09-23 18:30 ` Dumitrescu, Cristian 2020-09-29 13:51 ` David Marchand 1 sibling, 1 reply; 329+ messages in thread From: Stephen Hemminger @ 2020-09-23 18:26 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, thomas, david.marchand On Wed, 23 Sep 2020 19:06:40 +0100 Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > create mode 100644 examples/pipeline/Makefile Make is no longer supported, examples need to only use meson. ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application 2020-09-23 18:26 ` Stephen Hemminger @ 2020-09-23 18:30 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-23 18:30 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev, thomas, david.marchand > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Wednesday, September 23, 2020 7:26 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev@dpdk.org; thomas@monjalon.net; david.marchand@redhat.com > Subject: Re: [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new > example application > > On Wed, 23 Sep 2020 19:06:40 +0100 > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > create mode 100644 examples/pipeline/Makefile > > Make is no longer supported, examples need to only use meson. Hi Stephen, Not really, if you look in the latest DPDK tree, you'll see that every example app has a Makefile. We still want to allow people to build the example apps with Make. The Makefile support has been removed for the libraries only. Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-23 18:26 ` Stephen Hemminger @ 2020-09-29 13:51 ` David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 1 sibling, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-29 13:51 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon On Wed, Sep 23, 2020 at 8:07 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > Add new example application to showcase the API of the newly > introduced SWX pipeline type. > > Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> > --- > MAINTAINERS | 1 + > examples/meson.build | 1 + > examples/pipeline/Makefile | 51 ++++ > examples/pipeline/main.c | 50 ++++ > examples/pipeline/meson.build | 16 + > examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ > examples/pipeline/obj.h | 131 ++++++++ > examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ > examples/pipeline/thread.h | 28 ++ > 9 files changed, 1297 insertions(+) > create mode 100644 examples/pipeline/Makefile > create mode 100644 examples/pipeline/main.c > create mode 100644 examples/pipeline/meson.build > create mode 100644 examples/pipeline/obj.c > create mode 100644 examples/pipeline/obj.h > create mode 100644 examples/pipeline/thread.c > create mode 100644 examples/pipeline/thread.h > > diff --git a/MAINTAINERS b/MAINTAINERS > index 3b16d7a4b..ba8d55c22 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1332,6 +1332,7 @@ F: app/test/test_table* > F: app/test-pipeline/ > F: doc/guides/sample_app_ug/test_pipeline.rst > F: examples/ip_pipeline/ > +F: examples/pipeline/ Not really happy to see a new example which seems a clone of the old example. > F: doc/guides/sample_app_ug/ip_pipeline.rst > > > diff --git a/examples/meson.build b/examples/meson.build > index eb13e8210..245d98575 100644 > --- a/examples/meson.build > +++ b/examples/meson.build > @@ -33,6 +33,7 @@ all_examples = [ > 'ntb', 'packet_ordering', > 'performance-thread/l3fwd-thread', > 'performance-thread/pthread_shim', > + 'pipeline', > 'ptpclient', > 'qos_meter', 'qos_sched', > 'rxtx_callbacks', > diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile > new file mode 100644 > index 000000000..8d01fbfed > --- /dev/null > +++ b/examples/pipeline/Makefile > @@ -0,0 +1,51 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2010-2020 Intel Corporation Copyright 2020. > + > +# binary name > +APP = pipeline > + > +# all source are stored in SRCS-y > +SRCS-y += main.c > +SRCS-y += obj.c > +SRCS-y += thread.c > + > +# Build using pkg-config variables if possible > +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) > +$(error "no installation of DPDK found") > +endif > + > +all: shared > +.PHONY: shared static > +shared: build/$(APP)-shared > + ln -sf $(APP)-shared build/$(APP) > +static: build/$(APP)-static > + ln -sf $(APP)-static build/$(APP) > + > +PKGCONF ?= pkg-config > + > +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) > +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) I did not catch it on the first pass, but now looking deeper, this example uses experimental APIs and its makefile should contain the -DALLOW_EXPERIMENTAL_API cflag to indicate acceptance. This triggers build warnings (or errors with -Werror), when compiled as an external example. obj.c: In function ‘pipeline_create’: obj.c:381:2: warning: ‘rte_swx_pipeline_config’ is deprecated: Symbol is not yet part of stable ABI [-Wdeprecated-declarations] 381 | status = rte_swx_pipeline_config(&p, numa_node); | ^~~~~~ etc... > +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) > +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) Copied/pasted from older Makefiles ? I get a: ln -sf pipeline-shared build/pipeline cc build/main.o build/obj.o build/thread.o -o build/pipeline-static -L/home/dmarchan/intel-ipsec-mb/install/lib -Wl,-Bstatic -Wl,--whole-archive -L/home/dmarchan/builds/build-x86-default/install/usr/local/lib -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_pmd_af_packet.a -l:librte_pmd_af_xdp.a -l:librte_pmd_ark.a -l:librte_pmd_atlantic.a -l:librte_pmd_avp.a -l:librte_pmd_axgbe.a -l:librte_pmd_bond.a -l:librte_pmd_bnx2x.a -l:librte_pmd_bnxt.a -l:librte_pmd_cxgbe.a -l:librte_pmd_dpaa.a -l:librte_pmd_dpaa2.a -l:librte_pmd_e1000.a -l:librte_pmd_ena.a -l:librte_pmd_enetc.a -l:librte_pmd_enic.a -l:librte_pmd_failsafe.a -l:librte_pmd_fm10k.a -l:librte_pmd_i40e.a -l:librte_pmd_hinic.a -l:librte_pmd_hns3.a -l:librte_pmd_iavf.a -l:librte_pmd_ice.a -l:librte_pmd_igc.a -l:librte_pmd_ipn3ke.a -l:librte_pmd_ixgbe.a -l:librte_pmd_kni.a -l:librte_pmd_liquidio.a -l:librte_pmd_memif.a -l:librte_pmd_mlx4.a -l:librte_pmd_mlx5.a -l:librte_pmd_netvsc.a -l:librte_pmd_nfp.a -l:librte_pmd_null.a -l:librte_pmd_octeontx.a -l:librte_pmd_octeontx2.a -l:librte_pmd_pcap.a -l:librte_pmd_pfe.a -l:librte_pmd_qede.a -l:librte_pmd_ring.a -l:librte_pmd_sfc.a -l:librte_pmd_softnic.a -l:librte_pmd_szedata2.a -l:librte_pmd_tap.a -l:librte_pmd_thunderx.a -l:librte_pmd_vdev_netvsc.a -l:librte_pmd_vhost.a -l:librte_pmd_virtio.a -l:librte_pmd_vmxnet3.a -l:librte_rawdev_dpaa2_cmdif.a -l:librte_rawdev_dpaa2_qdma.a -l:librte_rawdev_ifpga.a -l:librte_rawdev_ioat.a -l:librte_rawdev_ntb.a -l:librte_rawdev_octeontx2_dma.a -l:librte_rawdev_octeontx2_ep.a -l:librte_rawdev_skeleton.a -l:librte_pmd_aesni_gcm.a -l:librte_pmd_aesni_mb.a -l:librte_pmd_caam_jr.a -l:librte_pmd_ccp.a -l:librte_pmd_dpaa_sec.a -l:librte_pmd_dpaa2_sec.a -l:librte_pmd_kasumi.a -l:librte_pmd_nitrox.a -l:librte_pmd_null_crypto.a -l:librte_pmd_octeontx_crypto.a -l:librte_pmd_octeontx2_crypto.a -l:librte_pmd_openssl.a -l:librte_pmd_crypto_scheduler.a -l:librte_pmd_snow3g.a -l:librte_pmd_virtio_crypto.a -l:librte_pmd_zuc.a -l:librte_pmd_isal.a -l:librte_pmd_octeontx_compress.a -l:librte_pmd_qat.a -l:librte_pmd_zlib.a -l:librte_pmd_mlx5_regex.a -l:librte_pmd_ifc.a -l:librte_pmd_mlx5_vdpa.a -l:librte_pmd_dpaa_event.a -l:librte_pmd_dpaa2_event.a -l:librte_pmd_octeontx2_event.a -l:librte_pmd_opdl_event.a -l:librte_pmd_skeleton_event.a -l:librte_pmd_sw_event.a -l:librte_pmd_dsw_event.a -l:librte_pmd_octeontx_event.a -l:librte_pmd_bbdev_null.a -l:librte_pmd_bbdev_turbo_sw.a -l:librte_pmd_bbdev_fpga_lte_fec.a -l:librte_pmd_bbdev_fpga_5gnr_fec.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lIPSec_MB -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lfdt -lpcap -lbsd -L/usr/usr/lib64 -lmlx5 -lpthread -L/usr/usr/lib64 -lpthread -libverbs -lpthread -lbpf -lz -lmlx4 -lpthread -L/usr/usr/lib64 -libverbs -lpthread -Wl,-R/usr/lib64 -lsze2 -lcrypto -lz -ldl -pthread -L/opt/isa-l/lib -lisal -lelf -lz -ljansson /usr/bin/ld: cannot find -lpcap Probably because of -Wl,-Bstatic. I suppose you must align with Bruce previous work: 8549295db07b ("build/pkg-config: improve static linking flags") (note for self, l3fwd-graph has to be looked at again.. suspect the same issue). > + > +CFLAGS += -I. > + > +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) > + > +build/%.o: %.c Makefile $(PC_FILE) | build > + $(CC) $(CFLAGS) -c $< -o $@ > + > +build/$(APP)-shared: $(OBJS) > + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) > + > +build/$(APP)-static: $(OBJS) > + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) > + > +build: > + @mkdir -p $@ > + > +.PHONY: clean > +clean: > + rm -f build/$(APP)* build/*.o > + test -d build && rmdir -p build || true > + Unneeded empty line. -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application 2020-09-29 13:51 ` David Marchand @ 2020-09-30 6:50 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-30 6:50 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon Hi David, > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Tuesday, September 29, 2020 2:51 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [PATCH v5 36/41] examples/pipeline: add new example > application > > On Wed, Sep 23, 2020 at 8:07 PM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > > > > Add new example application to showcase the API of the newly > > introduced SWX pipeline type. > > > > Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> > > --- > > MAINTAINERS | 1 + > > examples/meson.build | 1 + > > examples/pipeline/Makefile | 51 ++++ > > examples/pipeline/main.c | 50 ++++ > > examples/pipeline/meson.build | 16 + > > examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ > > examples/pipeline/obj.h | 131 ++++++++ > > examples/pipeline/thread.c | 549 > ++++++++++++++++++++++++++++++++++ > > examples/pipeline/thread.h | 28 ++ > > 9 files changed, 1297 insertions(+) > > create mode 100644 examples/pipeline/Makefile > > create mode 100644 examples/pipeline/main.c > > create mode 100644 examples/pipeline/meson.build > > create mode 100644 examples/pipeline/obj.c > > create mode 100644 examples/pipeline/obj.h > > create mode 100644 examples/pipeline/thread.c > > create mode 100644 examples/pipeline/thread.h > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 3b16d7a4b..ba8d55c22 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -1332,6 +1332,7 @@ F: app/test/test_table* > > F: app/test-pipeline/ > > F: doc/guides/sample_app_ug/test_pipeline.rst > > F: examples/ip_pipeline/ > > +F: examples/pipeline/ > > Not really happy to see a new example which seems a clone of the old > example. > The old example will be removed when the existing API (currently stable) will be deprecated and removed and the new API (currently experimental) will mature and become stable, as outlined in the cover letter. > > > F: doc/guides/sample_app_ug/ip_pipeline.rst > > > > > > diff --git a/examples/meson.build b/examples/meson.build > > index eb13e8210..245d98575 100644 > > --- a/examples/meson.build > > +++ b/examples/meson.build > > @@ -33,6 +33,7 @@ all_examples = [ > > 'ntb', 'packet_ordering', > > 'performance-thread/l3fwd-thread', > > 'performance-thread/pthread_shim', > > + 'pipeline', > > 'ptpclient', > > 'qos_meter', 'qos_sched', > > 'rxtx_callbacks', > > diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile > > new file mode 100644 > > index 000000000..8d01fbfed > > --- /dev/null > > +++ b/examples/pipeline/Makefile > > @@ -0,0 +1,51 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(c) 2010-2020 Intel Corporation > > Copyright 2020. > Changed in V6. > > + > > +# binary name > > +APP = pipeline > > + > > +# all source are stored in SRCS-y > > +SRCS-y += main.c > > +SRCS-y += obj.c > > +SRCS-y += thread.c > > + > > +# Build using pkg-config variables if possible > > +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) > > +$(error "no installation of DPDK found") > > +endif > > + > > +all: shared > > +.PHONY: shared static > > +shared: build/$(APP)-shared > > + ln -sf $(APP)-shared build/$(APP) > > +static: build/$(APP)-static > > + ln -sf $(APP)-static build/$(APP) > > + > > +PKGCONF ?= pkg-config > > + > > +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) > > +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) > > I did not catch it on the first pass, but now looking deeper, this > example uses experimental APIs and its makefile should contain the > -DALLOW_EXPERIMENTAL_API cflag to indicate acceptance. > Makefile fixed in V6, thanks! > This triggers build warnings (or errors with -Werror), when compiled > as an external example. > > > obj.c: In function ‘pipeline_create’: > obj.c:381:2: warning: ‘rte_swx_pipeline_config’ is deprecated: Symbol > is not yet part of stable ABI [-Wdeprecated-declarations] > 381 | status = rte_swx_pipeline_config(&p, numa_node); > | ^~~~~~ > etc... > > > > +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) > > +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) > > Copied/pasted from older Makefiles ? > Looks like Makefiles in examples changed since V4 or V5. Fixed in V6. > I get a: > > ln -sf pipeline-shared build/pipeline > cc build/main.o build/obj.o build/thread.o -o build/pipeline-static > -L/home/dmarchan/intel-ipsec-mb/install/lib -Wl,-Bstatic > -Wl,--whole-archive > -L/home/dmarchan/builds/build-x86-default/install/usr/local/lib > -l:librte_common_cpt.a -l:librte_common_dpaax.a > -l:librte_common_iavf.a -l:librte_common_octeontx.a > -l:librte_common_octeontx2.a -l:librte_bus_dpaa.a > -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a > -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a > -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a > -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a > -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a > -l:librte_mempool_stack.a -l:librte_pmd_af_packet.a > -l:librte_pmd_af_xdp.a -l:librte_pmd_ark.a -l:librte_pmd_atlantic.a > -l:librte_pmd_avp.a -l:librte_pmd_axgbe.a -l:librte_pmd_bond.a > -l:librte_pmd_bnx2x.a -l:librte_pmd_bnxt.a -l:librte_pmd_cxgbe.a > -l:librte_pmd_dpaa.a -l:librte_pmd_dpaa2.a -l:librte_pmd_e1000.a > -l:librte_pmd_ena.a -l:librte_pmd_enetc.a -l:librte_pmd_enic.a > -l:librte_pmd_failsafe.a -l:librte_pmd_fm10k.a -l:librte_pmd_i40e.a > -l:librte_pmd_hinic.a -l:librte_pmd_hns3.a -l:librte_pmd_iavf.a > -l:librte_pmd_ice.a -l:librte_pmd_igc.a -l:librte_pmd_ipn3ke.a > -l:librte_pmd_ixgbe.a -l:librte_pmd_kni.a -l:librte_pmd_liquidio.a > -l:librte_pmd_memif.a -l:librte_pmd_mlx4.a -l:librte_pmd_mlx5.a > -l:librte_pmd_netvsc.a -l:librte_pmd_nfp.a -l:librte_pmd_null.a > -l:librte_pmd_octeontx.a -l:librte_pmd_octeontx2.a > -l:librte_pmd_pcap.a -l:librte_pmd_pfe.a -l:librte_pmd_qede.a > -l:librte_pmd_ring.a -l:librte_pmd_sfc.a -l:librte_pmd_softnic.a > -l:librte_pmd_szedata2.a -l:librte_pmd_tap.a -l:librte_pmd_thunderx.a > -l:librte_pmd_vdev_netvsc.a -l:librte_pmd_vhost.a > -l:librte_pmd_virtio.a -l:librte_pmd_vmxnet3.a > -l:librte_rawdev_dpaa2_cmdif.a -l:librte_rawdev_dpaa2_qdma.a > -l:librte_rawdev_ifpga.a -l:librte_rawdev_ioat.a > -l:librte_rawdev_ntb.a -l:librte_rawdev_octeontx2_dma.a > -l:librte_rawdev_octeontx2_ep.a -l:librte_rawdev_skeleton.a > -l:librte_pmd_aesni_gcm.a -l:librte_pmd_aesni_mb.a > -l:librte_pmd_caam_jr.a -l:librte_pmd_ccp.a -l:librte_pmd_dpaa_sec.a > -l:librte_pmd_dpaa2_sec.a -l:librte_pmd_kasumi.a > -l:librte_pmd_nitrox.a -l:librte_pmd_null_crypto.a > -l:librte_pmd_octeontx_crypto.a -l:librte_pmd_octeontx2_crypto.a > -l:librte_pmd_openssl.a -l:librte_pmd_crypto_scheduler.a > -l:librte_pmd_snow3g.a -l:librte_pmd_virtio_crypto.a > -l:librte_pmd_zuc.a -l:librte_pmd_isal.a > -l:librte_pmd_octeontx_compress.a -l:librte_pmd_qat.a > -l:librte_pmd_zlib.a -l:librte_pmd_mlx5_regex.a -l:librte_pmd_ifc.a > -l:librte_pmd_mlx5_vdpa.a -l:librte_pmd_dpaa_event.a > -l:librte_pmd_dpaa2_event.a -l:librte_pmd_octeontx2_event.a > -l:librte_pmd_opdl_event.a -l:librte_pmd_skeleton_event.a > -l:librte_pmd_sw_event.a -l:librte_pmd_dsw_event.a > -l:librte_pmd_octeontx_event.a -l:librte_pmd_bbdev_null.a > -l:librte_pmd_bbdev_turbo_sw.a -l:librte_pmd_bbdev_fpga_lte_fec.a > -l:librte_pmd_bbdev_fpga_5gnr_fec.a -l:librte_node.a -l:librte_graph.a > -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a > -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a > -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a > -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a > -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a > -l:librte_power.a -l:librte_member.a -l:librte_lpm.a > -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a > -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a > -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a > -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a > -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a > -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a > -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a > -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a > -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a > -l:librte_kvargs.a -Wl,--no-whole-archive -lpcap -lIPSec_MB > -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify > -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec > -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder > -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power > -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats > -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd > -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile > -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash > -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter > -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal > -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lfdt -lpcap > -lbsd -L/usr/usr/lib64 -lmlx5 -lpthread -L/usr/usr/lib64 -lpthread > -libverbs -lpthread -lbpf -lz -lmlx4 -lpthread -L/usr/usr/lib64 > -libverbs -lpthread -Wl,-R/usr/lib64 -lsze2 -lcrypto -lz -ldl -pthread > -L/opt/isa-l/lib -lisal -lelf -lz -ljansson > /usr/bin/ld: cannot find -lpcap > > Probably because of -Wl,-Bstatic. > I suppose you must align with Bruce previous work: 8549295db07b > ("build/pkg-config: improve static linking flags") > Fixed in V6. > (note for self, l3fwd-graph has to be looked at again.. suspect the same > issue). > > > > + > > +CFLAGS += -I. > > + > > +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) > > + > > +build/%.o: %.c Makefile $(PC_FILE) | build > > + $(CC) $(CFLAGS) -c $< -o $@ > > + > > +build/$(APP)-shared: $(OBJS) > > + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) > > + > > +build/$(APP)-static: $(OBJS) > > + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) > > + > > +build: > > + @mkdir -p $@ > > + > > +.PHONY: clean > > +clean: > > + rm -f build/$(APP)* build/*.o > > + test -d build && rmdir -p build || true > > + > > Unneeded empty line. Removed in V6. > > > -- > David Marchand Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 37/41] examples/pipeline: add message passing mechanism 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu ` (4 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 137 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 519 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 8d01fbfed..2cb5edc1a 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dec78fba5..dc5a72899 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,5 +167,19 @@ main(int argc, char **argv) NULL, SKIP_MASTER); - return 0; + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; + + /* Dispatch loop */ + for ( ; ; ) { + conn_poll_for_conn(conn); + + conn_poll_for_msg(conn); + } } diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-29 13:51 ` David Marchand 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (3 subsequent siblings) 41 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 10 +- examples/pipeline/meson.build | 1 + 5 files changed, 1430 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 2cb5edc1a..ff32ad19b 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..7a1863ee7 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dc5a72899..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-29 13:51 ` David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 0 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-29 13:51 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon Caught while diffing with ip_pipeline code. On Wed, Sep 23, 2020 at 8:07 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: [snip] > + snprintf(out, out_size, > + "\n" > + "%s: flags=<%s> mtu %u\n" > + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" > + "\tport# %u speed %u Mbps\n" + "\tport# %u speed %s\n" > + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" > + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" > + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" > + "\tTX errors %" PRIu64"\n", > + link->name, > + eth_link.link_status == 0 ? "DOWN" : "UP", > + mtu, > + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], > + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], > + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], > + link->n_rxq, > + link->n_txq, > + link->port_id, > + eth_link.link_speed, + rte_eth_link_speed_to_str(eth_link.link_speed), > + stats.ipackets, > + stats.ibytes, > + stats.ierrors, > + stats.imissed, > + stats.rx_nombuf, > + stats.opackets, > + stats.obytes, > + stats.oerrors); -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands 2020-09-29 13:51 ` David Marchand @ 2020-09-30 6:50 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-30 6:50 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Tuesday, September 29, 2020 2:52 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [PATCH v5 38/41] examples/pipeline: add configuration > commands > > Caught while diffing with ip_pipeline code. > > On Wed, Sep 23, 2020 at 8:07 PM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > [snip] > > + snprintf(out, out_size, > > + "\n" > > + "%s: flags=<%s> mtu %u\n" > > + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u > txqueues %u\n" > > + "\tport# %u speed %u Mbps\n" > > + "\tport# %u speed %s\n" > > > + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" > > + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" > PRIu64"\n" > > + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" > > + "\tTX errors %" PRIu64"\n", > > + link->name, > > + eth_link.link_status == 0 ? "DOWN" : "UP", > > + mtu, > > + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], > > + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], > > + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], > > + link->n_rxq, > > + link->n_txq, > > + link->port_id, > > + eth_link.link_speed, > > + rte_eth_link_speed_to_str(eth_link.link_speed), > > > + stats.ipackets, > > + stats.ibytes, > > + stats.ierrors, > > + stats.imissed, > > + stats.rx_nombuf, > > + stats.opackets, > > + stats.obytes, > > + stats.oerrors); > > > > -- > David Marchand Using rte_eth_link_speed_to_str() in V6, thanks! ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 39/41] examples/pipeline: add l2fwd example 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu ` (2 subsequent siblings) 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..c6a3b9d04 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..be7773b58 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 40/41] examples/pipeline: add l2fwd with MAC swap example 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-09-29 14:08 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language David Marchand 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..8031b2655 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..9044d7d7f --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v5 41/41] examples/pipeline: add VXLAN encapsulation example 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-09-23 18:06 ` Cristian Dumitrescu 2020-09-29 14:08 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language David Marchand 41 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-23 18:06 UTC (permalink / raw) To: dev; +Cc: thomas, david.marchand Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..f1efd177e --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/pipeline/examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/pipeline/examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..aaa105da7 --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to headers and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c6975343e --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (40 preceding siblings ...) 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu @ 2020-09-29 14:08 ` David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 41 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-29 14:08 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon On Wed, Sep 23, 2020 at 8:06 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > This patch set introduces a new pipeline type that combines the DPDK > performance with the flexibility of the P4-16 language[1]. The new API > can be used either by itself to code a complete software switch (SWX) > or data plane app, or in combination with the open-source P4 compiler > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > programs to be translated to the DPDK API and run on multi-core CPUs. > > Main new features: > > * Nothing is hard-wired, everything is dynamically defined: The packet > headers (i.e. protocols), the packet meta-data, the actions, the > tables and the pipeline itself are dynamically defined instead of > having to be selected from a pre-defined set. > > * Instructions: The actions and the life of the packet through the > pipeline are defined with instructions that manipulate the pipeline > objects mentioned above. The pipeline is the main function of the > packet program, with actions as subroutines triggered by the tables. > > * Call external plugins: Extern objects and functions can be defined > to call functionality that cannot be efficiently implemented with > the existing pipeline-oriented instruction set, such as: special > error detecting/correcting codes, crypto, meters, stats arrays, > heuristics, etc. > > * Better control plane interaction: Transaction-oriented table update > mechanism that supports multi-table atomic updates. Multiple tables > can be updated in a single step with only the before and after table > sets visible to the packets. Alignment with P4Runtime [3]. > > * Performance: Multiple packets are in-flight within the pipeline at > any moment. Each packet is owned by a different time-sharing thread > in run-to-completion, with the thread pausing before memory access > operations such as packet I/O and table lookup to allow the memory > prefetch to complete. The instructions are verified and translated > at initialization time with no run-time impact. The instructions are > also optimized to detect and "fuse" frequently used patterns into > vector-like instructions transparently to the user. > > API deprecation and maturing roadmap: > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > prior to and removed as part of the DPDK 21.11 LTS release. > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > and become stable as part of the same DPDK 21.11 LTS release. This is a new feature: we are missing a release note update as part of the series. -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language 2020-09-29 14:08 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language David Marchand @ 2020-09-30 6:50 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-30 6:50 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Tuesday, September 29, 2020 3:09 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [PATCH v5 00/41] Pipeline alignment with the P4 language > > On Wed, Sep 23, 2020 at 8:06 PM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > > > > This patch set introduces a new pipeline type that combines the DPDK > > performance with the flexibility of the P4-16 language[1]. The new API > > can be used either by itself to code a complete software switch (SWX) > > or data plane app, or in combination with the open-source P4 compiler > > P4C [2], potentially acting as a P4C back-end, thus allowing the P4 > > programs to be translated to the DPDK API and run on multi-core CPUs. > > > > Main new features: > > > > * Nothing is hard-wired, everything is dynamically defined: The packet > > headers (i.e. protocols), the packet meta-data, the actions, the > > tables and the pipeline itself are dynamically defined instead of > > having to be selected from a pre-defined set. > > > > * Instructions: The actions and the life of the packet through the > > pipeline are defined with instructions that manipulate the pipeline > > objects mentioned above. The pipeline is the main function of the > > packet program, with actions as subroutines triggered by the tables. > > > > * Call external plugins: Extern objects and functions can be defined > > to call functionality that cannot be efficiently implemented with > > the existing pipeline-oriented instruction set, such as: special > > error detecting/correcting codes, crypto, meters, stats arrays, > > heuristics, etc. > > > > * Better control plane interaction: Transaction-oriented table update > > mechanism that supports multi-table atomic updates. Multiple tables > > can be updated in a single step with only the before and after table > > sets visible to the packets. Alignment with P4Runtime [3]. > > > > * Performance: Multiple packets are in-flight within the pipeline at > > any moment. Each packet is owned by a different time-sharing thread > > in run-to-completion, with the thread pausing before memory access > > operations such as packet I/O and table lookup to allow the memory > > prefetch to complete. The instructions are verified and translated > > at initialization time with no run-time impact. The instructions are > > also optimized to detect and "fuse" frequently used patterns into > > vector-like instructions transparently to the user. > > > > API deprecation and maturing roadmap: > > * The existing pipeline stable API (rte_pipeline.h) to be deprecated > > prior to and removed as part of the DPDK 21.11 LTS release. > > * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature > > and become stable as part of the same DPDK 21.11 LTS release. > > This is a new feature: we are missing a release note update as part of > the series. > > > -- > David Marchand Added release notes update in V6, thanks! ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 02/41] pipeline: add SWX pipeline input port 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (40 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 5 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 03/41] pipeline: add SWX pipeline output port 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (39 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 04/41] pipeline: add SWX headers and meta-data 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (38 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 05/41] pipeline: add SWX extern objects and funcs 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu ` (37 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 5 files changed, 694 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 06/41] pipeline: add SWX pipeline action 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (36 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 07/41] pipeline: add SWX pipeline tables 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (35 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..dc434b72e --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to facilitate the membership of this table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation searches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 08/41] pipeline: add SWX pipeline instructions 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu ` (34 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 09/41] pipeline: add SWX rx and extract instructions 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu ` (33 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add packet reception and header extraction instructions. The RX must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 10/41] pipeline: add SWX tx and emit instructions 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (32 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. TX ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 11/41] pipeline: add header validate and invalidate SWX instructions 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu ` (31 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 12/41] pipeline: add SWX mov instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu ` (30 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 13/41] pipeline: add SWX dma instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (29 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 14/41] pipeline: introduce SWX add instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu ` (28 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 15/41] pipeline: introduce SWX sub instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (27 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 16/41] pipeline: introduce SWX ckadd instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (26 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 17/41] pipeline: introduce SWX cksub instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (25 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 18/41] pipeline: introduce SWX and instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (24 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 19/41] pipeline: introduce SWX or instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu ` (23 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 20/41] pipeline: introduce SWX xor instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu ` (22 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 21/41] pipeline: introduce SWX shl instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu ` (21 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 22/41] pipeline: introduce SWX shr instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (20 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 23/41] pipeline: introduce SWX table instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (19 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 24/41] pipeline: introduce SWX extern instruction 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu ` (18 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance to the mailbox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 25/41] pipeline: introduce SWX jmp and return instructions 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 26/41] pipeline: add SWX instruction description Cristian Dumitrescu ` (17 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 26/41] pipeline: add SWX instruction description 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (16 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..d6c086e27 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 27/41] pipeline: add SWX instruction verifier 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 26/41] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (15 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 28/41] pipeline: add SWX instruction optimizer 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (14 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 29/41] pipeline: add SWX pipeline query API 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (13 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..344c7c833 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup operation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 30/41] pipeline: add SWX pipeline flush 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu ` (12 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d6c086e27..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 31/41] pipeline: add SWX table update high level API 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (11 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 4 files changed, 1736 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..576fb2bf3 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wildcard and LPM tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index 344c7c833..18b065834 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 32/41] pipeline: add SWX pipeline specification file 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 33/41] port: add ethernet device SWX port Cristian Dumitrescu ` (10 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 6da5710af..6928e78b6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 33/41] port: add ethernet device SWX port 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 34/41] port: add source and sink SWX ports Cristian Dumitrescu ` (9 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 34/41] port: add source and sink SWX ports 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 33/41] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 35/41] table: add exact match SWX table Cristian Dumitrescu ` (8 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 35/41] table: add exact match SWX table 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 34/41] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 36/41] examples/pipeline: add new example application Cristian Dumitrescu ` (7 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 4 files changed, 892 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 36/41] examples/pipeline: add new example application 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 35/41] table: add exact match SWX table Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (6 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/meson.build | 1 + examples/pipeline/Makefile | 51 ++++ examples/pipeline/main.c | 50 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 8 files changed, 1296 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..8d01fbfed --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..dec78fba5 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + return 0; +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..688870f97 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..1be9828f0 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Control thread: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Control thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Control thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Control thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Control thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 37/41] examples/pipeline: add message passing mechanism 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 36/41] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu ` (5 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 137 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 519 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 8d01fbfed..2cb5edc1a 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dec78fba5..dc5a72899 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,5 +167,19 @@ main(int argc, char **argv) NULL, SKIP_MASTER); - return 0; + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; + + /* Dispatch loop */ + for ( ; ; ) { + conn_poll_for_conn(conn); + + conn_poll_for_msg(conn); + } } diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 38/41] examples/pipeline: add configuration commands 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (4 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 10 +- examples/pipeline/meson.build | 1 + 5 files changed, 1430 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 2cb5edc1a..ff32ad19b 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..7a1863ee7 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index dc5a72899..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 39/41] examples/pipeline: add l2fwd example 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu ` (3 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..c6a3b9d04 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..be7773b58 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 40/41] examples/pipeline: add l2fwd with MAC swap example 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu ` (2 subsequent siblings) 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..8031b2655 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..9044d7d7f --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v4 41/41] examples/pipeline: add VXLAN encapsulation example 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-09-10 15:26 ` Cristian Dumitrescu 2020-09-22 20:05 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Wang, Han2 2020-09-23 11:45 ` David Marchand 42 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-10 15:26 UTC (permalink / raw) To: dev Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..f1efd177e --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/pipeline/examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/pipeline/examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..aaa105da7 --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to headers and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c6975343e --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (40 preceding siblings ...) 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu @ 2020-09-22 20:05 ` Wang, Han2 2020-09-22 20:08 ` Dumitrescu, Cristian 2020-09-23 11:45 ` David Marchand 42 siblings, 1 reply; 329+ messages in thread From: Wang, Han2 @ 2020-09-22 20:05 UTC (permalink / raw) To: Dumitrescu, Cristian, dev Cristian Thanks for updating the patch set. As I mentioned in my previous email, we are working on adding support for this as a new P4C CPU target. We plan to open source the compiler backend in the near future. Cheers, Han On 9/10/20, 8:27 AM, "dev on behalf of Cristian Dumitrescu" <dev-bounces@dpdk.org on behalf of cristian.dumitrescu@intel.com> wrote: This patch set introduces a new pipeline type that combines the DPDK performance with the flexibility of the P4-16 language[1]. The new API can be used either by itself to code a complete software switch (SWX) or data plane app, or in combination with the open-source P4 compiler P4C [2], potentially acting as a P4C back-end, thus allowing the P4 programs to be translated to the DPDK API and run on multi-core CPUs. Main new features: * Nothing is hard-wired, everything is dynamically defined: The packet headers (i.e. protocols), the packet meta-data, the actions, the tables and the pipeline itself are dynamically defined instead of having to be selected from a pre-defined set. * Instructions: The actions and the life of the packet through the pipeline are defined with instructions that manipulate the pipeline objects mentioned above. The pipeline is the main function of the packet program, with actions as subroutines triggered by the tables. * Call external plugins: Extern objects and functions can be defined to call functionality that cannot be efficiently implemented with the existing pipeline-oriented instruction set, such as: special error detecting/correcting codes, crypto, meters, stats arrays, heuristics, etc. * Better control plane interaction: Transaction-oriented table update mechanism that supports multi-table atomic updates. Multiple tables can be updated in a single step with only the before and after table sets visible to the packets. Alignment with P4Runtime [3]. * Performance: Multiple packets are in-flight within the pipeline at any moment. Each packet is owned by a different time-sharing thread in run-to-completion, with the thread pausing before memory access operations such as packet I/O and table lookup to allow the memory prefetch to complete. The instructions are verified and translated at initialization time with no run-time impact. The instructions are also optimized to detect and "fuse" frequently used patterns into vector-like instructions transparently to the user. API deprecation and maturing roadmap: * The existing pipeline stable API (rte_pipeline.h) to be deprecated prior to and removed as part of the DPDK 21.11 LTS release. * The new SWX pipeline experimental API (rte_swx_pipeline.h) to mature and become stable as part of the same DPDK 21.11 LTS release. V4 changes: * Spell check fixes. V3 changes: * Removed the library Makefile support to align with the latest DPDK. V2 changes: * Updated the title and commit messages to reflect the introduction of the new SWX pipeline type. * Added the API deprecation and maturing roadmap to the cover letter. * Added support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The spec file may be generated by the P4C compiler in the future (see patch 32). Reworked the examples accordingly (see patches 39, 40 and 41). * Added support for the SWX sink port (used for packet drop or log) when PCAP library is disabled from the build. * Added checks to the application CLI commands to prevent execution when dependencies of the current command have previously failed (see patch 38). * Fixed build warning for 32-bit targets due to the printing of 64-bit statistics counters (see patch 38). [1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf [2] P4-16 compiler: https://github.com/p4lang/p4c [3] P4Runtime specification: https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf Cristian Dumitrescu (41): pipeline: add new SWX pipeline type pipeline: add SWX pipeline input port pipeline: add SWX pipeline output port pipeline: add SWX headers and meta-data pipeline: add SWX extern objects and funcs pipeline: add SWX pipeline action pipeline: add SWX pipeline tables pipeline: add SWX pipeline instructions pipeline: add SWX rx and extract instructions pipeline: add SWX tx and emit instructions pipeline: add header validate and invalidate SWX instructions pipeline: add SWX mov instruction pipeline: add SWX dma instruction pipeline: introduce SWX add instruction pipeline: introduce SWX sub instruction pipeline: introduce SWX ckadd instruction pipeline: introduce SWX cksub instruction pipeline: introduce SWX and instruction pipeline: introduce SWX or instruction pipeline: introduce SWX xor instruction pipeline: introduce SWX shl instruction pipeline: introduce SWX shr instruction pipeline: introduce SWX table instruction pipeline: introduce SWX extern instruction pipeline: introduce SWX jmp and return instructions pipeline: add SWX instruction description pipeline: add SWX instruction verifier pipeline: add SWX instruction optimizer pipeline: add SWX pipeline query API pipeline: add SWX pipeline flush pipeline: add SWX table update high level API pipeline: add SWX pipeline specification file port: add ethernet device SWX port port: add source and sink SWX ports table: add exact match SWX table examples/pipeline: add new example application examples/pipeline: add message passing mechanism examples/pipeline: add configuration commands examples/pipeline: add l2fwd example examples/pipeline: add l2fwd with MAC swap example examples/pipeline: add VXLAN encapsulation example examples/meson.build | 1 + examples/pipeline/Makefile | 53 + examples/pipeline/cli.c | 1400 ++++ examples/pipeline/cli.h | 19 + examples/pipeline/conn.c | 331 + examples/pipeline/conn.h | 50 + examples/pipeline/examples/l2fwd.cli | 25 + examples/pipeline/examples/l2fwd.spec | 42 + examples/pipeline/examples/l2fwd_macswp.cli | 25 + examples/pipeline/examples/l2fwd_macswp.spec | 59 + .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 + examples/pipeline/examples/l2fwd_pcap.cli | 20 + examples/pipeline/examples/packet.txt | 102 + examples/pipeline/examples/vxlan.cli | 27 + examples/pipeline/examples/vxlan.spec | 173 + examples/pipeline/examples/vxlan_pcap.cli | 22 + examples/pipeline/examples/vxlan_table.py | 71 + examples/pipeline/examples/vxlan_table.txt | 16 + examples/pipeline/main.c | 193 + examples/pipeline/meson.build | 18 + examples/pipeline/obj.c | 470 ++ examples/pipeline/obj.h | 131 + examples/pipeline/thread.c | 549 ++ examples/pipeline/thread.h | 28 + lib/librte_pipeline/meson.build | 14 +- lib/librte_pipeline/rte_pipeline_version.map | 44 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++ lib/librte_pipeline/rte_swx_ctl.h | 568 ++ lib/librte_pipeline/rte_swx_extern.h | 98 + lib/librte_pipeline/rte_swx_pipeline.c | 7197 +++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 711 ++ lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++ lib/librte_port/meson.build | 9 +- lib/librte_port/rte_port_version.map | 5 +- lib/librte_port/rte_swx_port.h | 202 + lib/librte_port/rte_swx_port_ethdev.c | 313 + lib/librte_port/rte_swx_port_ethdev.h | 54 + lib/librte_port/rte_swx_port_source_sink.c | 335 + lib/librte_port/rte_swx_port_source_sink.h | 57 + lib/librte_table/meson.build | 7 +- lib/librte_table/rte_swx_table.h | 295 + lib/librte_table/rte_swx_table_em.c | 851 ++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 44 files changed, 17625 insertions(+), 8 deletions(-) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h create mode 100644 lib/librte_pipeline/rte_swx_ctl.c create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_pipeline/rte_swx_extern.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c create mode 100644 lib/librte_port/rte_swx_port.h create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h create mode 100644 lib/librte_table/rte_swx_table.h create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-22 20:05 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Wang, Han2 @ 2020-09-22 20:08 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-22 20:08 UTC (permalink / raw) To: Wang, Han2, dev > -----Original Message----- > From: Wang, Han2 <han2.wang@intel.com> > Sent: Tuesday, September 22, 2020 9:06 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 > language > > Cristian > > Thanks for updating the patch set. As I mentioned in my previous email, we > are working on adding support for this as a new P4C CPU target. We plan to > open source the compiler backend in the near future. > > Cheers, > Han > Hi Han, That is great news, thanks very much for your work! Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (41 preceding siblings ...) 2020-09-22 20:05 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Wang, Han2 @ 2020-09-23 11:45 ` David Marchand 2020-09-23 16:07 ` Dumitrescu, Cristian 42 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-23 11:45 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev, Thomas Monjalon Hello Cristian, On Thu, Sep 10, 2020 at 5:27 PM Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > Cristian Dumitrescu (41): > pipeline: add new SWX pipeline type > pipeline: add SWX pipeline input port > pipeline: add SWX pipeline output port > pipeline: add SWX headers and meta-data > pipeline: add SWX extern objects and funcs > pipeline: add SWX pipeline action > pipeline: add SWX pipeline tables > pipeline: add SWX pipeline instructions > pipeline: add SWX rx and extract instructions > pipeline: add SWX tx and emit instructions > pipeline: add header validate and invalidate SWX instructions > pipeline: add SWX mov instruction > pipeline: add SWX dma instruction > pipeline: introduce SWX add instruction > pipeline: introduce SWX sub instruction > pipeline: introduce SWX ckadd instruction > pipeline: introduce SWX cksub instruction > pipeline: introduce SWX and instruction > pipeline: introduce SWX or instruction > pipeline: introduce SWX xor instruction > pipeline: introduce SWX shl instruction > pipeline: introduce SWX shr instruction > pipeline: introduce SWX table instruction > pipeline: introduce SWX extern instruction > pipeline: introduce SWX jmp and return instructions > pipeline: add SWX instruction description > pipeline: add SWX instruction verifier > pipeline: add SWX instruction optimizer > pipeline: add SWX pipeline query API > pipeline: add SWX pipeline flush > pipeline: add SWX table update high level API > pipeline: add SWX pipeline specification file > port: add ethernet device SWX port > port: add source and sink SWX ports > table: add exact match SWX table > examples/pipeline: add new example application > examples/pipeline: add message passing mechanism > examples/pipeline: add configuration commands > examples/pipeline: add l2fwd example > examples/pipeline: add l2fwd with MAC swap example > examples/pipeline: add VXLAN encapsulation example - This new feature is the future of the pipeline library: it will need unit tests and/or tests in the CI to catch regressions. - Many MACRO_WITH_FLOW_CONTROL warnings reported by checkpatches. - On the patch titles, check-git-log.sh reports: Wrong headline case: "pipeline: add SWX dma instruction": dma --> DMA Wrong headline case: "pipeline: add SWX rx and extract instructions": rx --> Rx Wrong headline case: "pipeline: add SWX tx and emit instructions": tx --> Tx Wrong headline case: "pipeline: introduce SWX xor instruction": xor --> XOR - We have yet another new example, please declare it in MAINTAINERS. - I checked per patch compilation which is fine, thanks. -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-23 11:45 ` David Marchand @ 2020-09-23 16:07 ` Dumitrescu, Cristian 2020-09-23 16:28 ` David Marchand 0 siblings, 1 reply; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-23 16:07 UTC (permalink / raw) To: David Marchand; +Cc: dev, Thomas Monjalon Hi David, Thanks for looking at this patch series! > -----Original Message----- > From: David Marchand <david.marchand@redhat.com> > Sent: Wednesday, September 23, 2020 12:46 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net> > Subject: Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 > language > > Hello Cristian, > > On Thu, Sep 10, 2020 at 5:27 PM Cristian Dumitrescu > <cristian.dumitrescu@intel.com> wrote: > > Cristian Dumitrescu (41): > > pipeline: add new SWX pipeline type > > pipeline: add SWX pipeline input port > > pipeline: add SWX pipeline output port > > pipeline: add SWX headers and meta-data > > pipeline: add SWX extern objects and funcs > > pipeline: add SWX pipeline action > > pipeline: add SWX pipeline tables > > pipeline: add SWX pipeline instructions > > pipeline: add SWX rx and extract instructions > > pipeline: add SWX tx and emit instructions > > pipeline: add header validate and invalidate SWX instructions > > pipeline: add SWX mov instruction > > pipeline: add SWX dma instruction > > pipeline: introduce SWX add instruction > > pipeline: introduce SWX sub instruction > > pipeline: introduce SWX ckadd instruction > > pipeline: introduce SWX cksub instruction > > pipeline: introduce SWX and instruction > > pipeline: introduce SWX or instruction > > pipeline: introduce SWX xor instruction > > pipeline: introduce SWX shl instruction > > pipeline: introduce SWX shr instruction > > pipeline: introduce SWX table instruction > > pipeline: introduce SWX extern instruction > > pipeline: introduce SWX jmp and return instructions > > pipeline: add SWX instruction description > > pipeline: add SWX instruction verifier > > pipeline: add SWX instruction optimizer > > pipeline: add SWX pipeline query API > > pipeline: add SWX pipeline flush > > pipeline: add SWX table update high level API > > pipeline: add SWX pipeline specification file > > port: add ethernet device SWX port > > port: add source and sink SWX ports > > table: add exact match SWX table > > examples/pipeline: add new example application > > examples/pipeline: add message passing mechanism > > examples/pipeline: add configuration commands > > examples/pipeline: add l2fwd example > > examples/pipeline: add l2fwd with MAC swap example > > examples/pipeline: add VXLAN encapsulation example > > - This new feature is the future of the pipeline library: it will need > unit tests and/or tests in the CI to catch regressions. > Agreed, this is work in progress and will materialize in new test cases upstreamed into DPDK Test Framework (DTS) during the 20.11 and 21.02 time frame. They are based on the new sample app introduced by this patch set, which already has some tests (see examples/pipeline/examples folder), but having them automated into DTS is WIP. > - Many MACRO_WITH_FLOW_CONTROL warnings reported by checkpatches. > Yes, I fixed all the other code style issues, this is the only one remaining. It is basically due to a recurring CHECK() macro. And it will also ripples over the code, so IMO it is time consuming & error prone to remove with no real benefit. We also already have many places in DPDK that use the same pattern. I suggest we ignore this warning, are you OK with it? > - On the patch titles, check-git-log.sh reports: > Wrong headline case: > "pipeline: add SWX dma instruction": dma --> DMA > Wrong headline case: > "pipeline: add SWX rx and extract instructions": rx --> Rx > Wrong headline case: > "pipeline: add SWX tx and emit instructions": tx --> Tx > Wrong headline case: > "pipeline: introduce SWX xor instruction": xor --> XOR > I can do this change, but IMO it is not the right choice here, as in this particular case we have instructions that are called "rx", "tx", "dma", "and", "or", "xor", etc. So it is the name of an instruction rather than a text abbreviation. Hence, I think these messages are not really applicable here. What do you think? > - We have yet another new example, please declare it in MAINTAINERS. > Yes, will fix in V5. > - I checked per patch compilation which is fine, thanks. > Great, thanks! > > -- > David Marchand Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-23 16:07 ` Dumitrescu, Cristian @ 2020-09-23 16:28 ` David Marchand 2020-09-23 16:40 ` Thomas Monjalon 0 siblings, 1 reply; 329+ messages in thread From: David Marchand @ 2020-09-23 16:28 UTC (permalink / raw) To: Dumitrescu, Cristian; +Cc: dev, Thomas Monjalon On Wed, Sep 23, 2020 at 6:08 PM Dumitrescu, Cristian <cristian.dumitrescu@intel.com> wrote: > > - Many MACRO_WITH_FLOW_CONTROL warnings reported by checkpatches. > > > > Yes, I fixed all the other code style issues, this is the only one remaining. It is basically due to a recurring CHECK() macro. And it will also ripples over the code, so IMO it is time consuming & error prone to remove with no real benefit. > > We also already have many places in DPDK that use the same pattern. I suggest we ignore this warning, are you OK with it? I am fine with ignoring, this is not like we have no other occurrence of such macros. I still see little value in those specific macros. > > - On the patch titles, check-git-log.sh reports: > > Wrong headline case: > > "pipeline: add SWX dma instruction": dma --> DMA > > Wrong headline case: > > "pipeline: add SWX rx and extract instructions": rx --> Rx > > Wrong headline case: > > "pipeline: add SWX tx and emit instructions": tx --> Tx > > Wrong headline case: > > "pipeline: introduce SWX xor instruction": xor --> XOR > > > > I can do this change, but IMO it is not the right choice here, as in this particular case we have instructions that are called "rx", "tx", "dma", "and", "or", "xor", etc. So it is the name of an instruction rather than a text abbreviation. Hence, I think these messages are not really applicable here. What do you think? For this reason I am ok with ignoring too, Thomas wdyt? -- David Marchand ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-23 16:28 ` David Marchand @ 2020-09-23 16:40 ` Thomas Monjalon 2020-09-23 16:49 ` Dumitrescu, Cristian 0 siblings, 1 reply; 329+ messages in thread From: Thomas Monjalon @ 2020-09-23 16:40 UTC (permalink / raw) To: Dumitrescu, Cristian, David Marchand; +Cc: dev 23/09/2020 18:28, David Marchand: > On Wed, Sep 23, 2020 at 6:08 PM Dumitrescu, Cristian > <cristian.dumitrescu@intel.com> wrote: > > > - On the patch titles, check-git-log.sh reports: > > > Wrong headline case: > > > "pipeline: add SWX dma instruction": dma --> DMA > > > Wrong headline case: > > > "pipeline: add SWX rx and extract instructions": rx --> Rx > > > Wrong headline case: > > > "pipeline: add SWX tx and emit instructions": tx --> Tx > > > Wrong headline case: > > > "pipeline: introduce SWX xor instruction": xor --> XOR > > > > > > > I can do this change, but IMO it is not the right choice here, as in this particular case we have instructions that are called "rx", "tx", "dma", "and", "or", "xor", etc. So it is the name of an instruction rather than a text abbreviation. Hence, I think these messages are not really applicable here. What do you think? > > For this reason I am ok with ignoring too, Thomas wdyt? The general idea of titles is to not use exact same wording as in code (no function or variable names for instance). For the instructions, I don't know. If you think it is better as lowercase, I can be convinced. ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-23 16:40 ` Thomas Monjalon @ 2020-09-23 16:49 ` Dumitrescu, Cristian 2020-09-23 19:02 ` Dumitrescu, Cristian 0 siblings, 1 reply; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-23 16:49 UTC (permalink / raw) To: Thomas Monjalon, David Marchand; +Cc: dev > -----Original Message----- > From: Thomas Monjalon <thomas@monjalon.net> > Sent: Wednesday, September 23, 2020 5:40 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; David Marchand > <david.marchand@redhat.com> > Cc: dev <dev@dpdk.org> > Subject: Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 > language > > 23/09/2020 18:28, David Marchand: > > On Wed, Sep 23, 2020 at 6:08 PM Dumitrescu, Cristian > > <cristian.dumitrescu@intel.com> wrote: > > > > - On the patch titles, check-git-log.sh reports: > > > > Wrong headline case: > > > > "pipeline: add SWX dma instruction": dma --> DMA > > > > Wrong headline case: > > > > "pipeline: add SWX rx and extract instructions": rx --> Rx > > > > Wrong headline case: > > > > "pipeline: add SWX tx and emit instructions": tx --> Tx > > > > Wrong headline case: > > > > "pipeline: introduce SWX xor instruction": xor --> XOR > > > > > > > > > > I can do this change, but IMO it is not the right choice here, as in this > particular case we have instructions that are called "rx", "tx", "dma", "and", > "or", "xor", etc. So it is the name of an instruction rather than a text > abbreviation. Hence, I think these messages are not really applicable here. > What do you think? > > > > For this reason I am ok with ignoring too, Thomas wdyt? > > The general idea of titles is to not use exact same wording as in code > (no function or variable names for instance). > For the instructions, I don't know. > If you think it is better as lowercase, I can be convinced. > OK, let me do the change in V5, thanks! ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language 2020-09-23 16:49 ` Dumitrescu, Cristian @ 2020-09-23 19:02 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-23 19:02 UTC (permalink / raw) To: Thomas Monjalon, David Marchand; +Cc: dev > -----Original Message----- > From: Dumitrescu, Cristian > Sent: Wednesday, September 23, 2020 5:49 PM > To: 'Thomas Monjalon' <thomas@monjalon.net>; David Marchand > <david.marchand@redhat.com> > Cc: dev <dev@dpdk.org> > Subject: RE: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 > language > > > > > -----Original Message----- > > From: Thomas Monjalon <thomas@monjalon.net> > > Sent: Wednesday, September 23, 2020 5:40 PM > > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; David Marchand > > <david.marchand@redhat.com> > > Cc: dev <dev@dpdk.org> > > Subject: Re: [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 > > language > > > > 23/09/2020 18:28, David Marchand: > > > On Wed, Sep 23, 2020 at 6:08 PM Dumitrescu, Cristian > > > <cristian.dumitrescu@intel.com> wrote: > > > > > - On the patch titles, check-git-log.sh reports: > > > > > Wrong headline case: > > > > > "pipeline: add SWX dma instruction": dma --> DMA > > > > > Wrong headline case: > > > > > "pipeline: add SWX rx and extract instructions": rx --> Rx > > > > > Wrong headline case: > > > > > "pipeline: add SWX tx and emit instructions": tx --> Tx > > > > > Wrong headline case: > > > > > "pipeline: introduce SWX xor instruction": xor --> XOR > > > > > > > > > > > > > I can do this change, but IMO it is not the right choice here, as in this > > particular case we have instructions that are called "rx", "tx", "dma", "and", > > "or", "xor", etc. So it is the name of an instruction rather than a text > > abbreviation. Hence, I think these messages are not really applicable here. > > What do you think? > > > > > > For this reason I am ok with ignoring too, Thomas wdyt? > > > > The general idea of titles is to not use exact same wording as in code > > (no function or variable names for instance). > > For the instructions, I don't know. > > If you think it is better as lowercase, I can be convinced. > > > > OK, let me do the change in V5, thanks! Done in V5 just sent, thanks! ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 02/41] pipeline: add SWX pipeline input port 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (38 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 5 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 03/41] pipeline: add SWX pipeline output port 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (37 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 04/41] pipeline: add SWX headers and meta-data 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (36 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 05/41] pipeline: add SWX extern objects and funcs 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu ` (35 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 5 files changed, 694 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 06/41] pipeline: add SWX pipeline action 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (34 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 07/41] pipeline: add SWX pipeline tables 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (33 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 7 files changed, 1206 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..c5c202723 --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to faciliate the addition of the current table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation seaches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 08/41] pipeline: add SWX pipeline instructions 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu ` (32 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 09/41] pipeline: add SWX rx and extract instructions 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu ` (31 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add packet reception and header extraction instructions. The RX must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 10/41] pipeline: add SWX tx and emit instructions 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu @ 2020-09-08 20:17 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (30 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:17 UTC (permalink / raw) To: dev Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. TX ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 11/41] pipeline: add header validate and invalidate SWX instructions 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu ` (29 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 12/41] pipeline: add SWX mov instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu ` (28 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 13/41] pipeline: add SWX dma instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (27 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 14/41] pipeline: introduce SWX add instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu ` (26 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 15/41] pipeline: introduce SWX sub instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (25 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 16/41] pipeline: introduce SWX ckadd instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (24 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 17/41] pipeline: introduce SWX cksub instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (23 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 18/41] pipeline: introduce SWX and instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (22 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 19/41] pipeline: introduce SWX or instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu ` (21 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 20/41] pipeline: introduce SWX xor instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu ` (20 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 21/41] pipeline: introduce SWX shl instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu ` (19 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 22/41] pipeline: introduce SWX shr instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (18 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 23/41] pipeline: introduce SWX table instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (17 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 24/41] pipeline: introduce SWX extern instruction 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu ` (16 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance in the maibox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 25/41] pipeline: introduce SWX jmp and return instructions 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 26/41] pipeline: add SWX instruction description Cristian Dumitrescu ` (15 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 26/41] pipeline: add SWX instruction description 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (14 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..d6c086e27 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 27/41] pipeline: add SWX instruction verifier 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 26/41] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (13 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 28/41] pipeline: add SWX instruction optimizer 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (12 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 29/41] pipeline: add SWX pipeline query API 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (11 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..bdcc24cee 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup opeation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 30/41] pipeline: add SWX pipeline flush 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu ` (10 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d6c086e27..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 31/41] pipeline: add SWX table update high level API 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (9 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 4 files changed, 1736 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..44da678c1 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wilcard and LPM match tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index bdcc24cee..786adedb2 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 32/41] pipeline: add SWX pipeline specification file 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 33/41] port: add ethernet device SWX port Cristian Dumitrescu ` (8 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 6da5710af..6928e78b6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 33/41] port: add ethernet device SWX port 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 34/41] port: add source and sink SWX ports Cristian Dumitrescu ` (7 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 4 files changed, 373 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 34/41] port: add source and sink SWX ports 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 33/41] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 35/41] table: add exact match SWX table Cristian Dumitrescu ` (6 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 35/41] table: add exact match SWX table 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 34/41] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 36/41] examples/pipeline: add new example application Cristian Dumitrescu ` (5 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 4 files changed, 892 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 36/41] examples/pipeline: add new example application 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 35/41] table: add exact match SWX table Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (4 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/meson.build | 1 + examples/pipeline/Makefile | 51 ++++ examples/pipeline/main.c | 52 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 8 files changed, 1298 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..8d01fbfed --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifneq ($(shell pkg-config --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..353e62c10 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + /* Dispatch loop */ + for ( ; ; ); +} + diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..688870f97 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..0d1474c55 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Master thead: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Master thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Master thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Master thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Master thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 37/41] examples/pipeline: add message passing mechanism 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 36/41] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu ` (3 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 136 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 517 insertions(+), 2 deletions(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 8d01fbfed..2cb5edc1a 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 353e62c10..1a63c1237 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,7 +167,18 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; /* Dispatch loop */ - for ( ; ; ); -} + for ( ; ; ) { + conn_poll_for_conn(conn); + conn_poll_for_msg(conn); + } +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 38/41] examples/pipeline: add configuration commands 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (2 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 11 +- examples/pipeline/meson.build | 1 + 5 files changed, 1431 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 2cb5edc1a..ff32ad19b 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..7a1863ee7 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 1a63c1237..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); @@ -175,6 +183,7 @@ main(int argc, char **argv) status); return status; }; + /* Dispatch loop */ for ( ; ; ) { conn_poll_for_conn(conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 39/41] examples/pipeline: add l2fwd example 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..c6a3b9d04 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..be7773b58 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 40/41] examples/pipeline: add l2fwd with MAC swap example 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..8031b2655 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/pipeline/examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..9044d7d7f --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v3 41/41] examples/pipeline: add VXLAN encapsulation example 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-09-08 20:18 ` Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-08 20:18 UTC (permalink / raw) To: dev Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..f1efd177e --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/pipeline/examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/pipeline/examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..47106471f --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to haders and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c6975343e --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 02/41] pipeline: add SWX pipeline input port 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu ` (38 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add input ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/Makefile | 1 + lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 6 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 57d2aedbc..4221618b3 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -55,5 +55,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 03/41] pipeline: add SWX pipeline output port 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu ` (37 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add output ports to the newly introduced SWX pipeline type. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 04/41] pipeline: add SWX headers and meta-data 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu ` (36 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add support for dynamically-defined packet headers and meta-data to the SWX pipeline. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 05/41] pipeline: add SWX extern objects and funcs 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu ` (35 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add extern objects and functions to plug into the SWX pipeline any functionality that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 6 files changed, 695 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 32582db9e..23bfd88e6 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 06/41] pipeline: add SWX pipeline action 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu ` (34 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add SWX actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 07/41] pipeline: add SWX pipeline tables 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu ` (33 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev Add tables to the SWX pipeline. The match fields are flexibly selected from the headers and meta-data. The set of table actions is flexibly selected for each table from the set of pipeline actions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/Makefile | 1 + lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 9 files changed, 1208 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 23bfd88e6..d214b1aeb 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -27,5 +27,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_ctl.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..eb5b327e8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + CHECK(hf || mf, EINVAL); + + offset = hf ? hf->offset : mf->offset; + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 6ad8a6b17..9df58698d 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -55,5 +55,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru_arm64.h endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..c5c202723 --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to faciliate the addition of the current table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation seaches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 08/41] pipeline: add SWX pipeline instructions 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu @ 2020-09-07 21:39 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu ` (32 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:39 UTC (permalink / raw) To: dev The SWX pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index eb5b327e8..2ae6229d0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 09/41] pipeline: add SWX rx and extract instructions 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu ` (31 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add packet reception and header extraction instructions. The RX must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2ae6229d0..d7af80e39 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 10/41] pipeline: add SWX tx and emit instructions 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu ` (30 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. TX ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d7af80e39..19bf2761d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 11/41] pipeline: add header validate and invalidate SWX instructions 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu ` (29 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 19bf2761d..8ddd766c2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 12/41] pipeline: add SWX mov instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu ` (28 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8ddd766c2..b5b502caa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 13/41] pipeline: add SWX dma instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu ` (27 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index b5b502caa..341afc735 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 14/41] pipeline: introduce SWX add instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu ` (26 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 341afc735..6eee52f24 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 15/41] pipeline: introduce SWX sub instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu ` (25 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6eee52f24..245621dc3 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 16/41] pipeline: introduce SWX ckadd instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu ` (24 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 245621dc3..96e6c98aa 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 17/41] pipeline: introduce SWX cksub instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu ` (23 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 96e6c98aa..364c7d75a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 18/41] pipeline: introduce SWX and instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu ` (22 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 364c7d75a..fe44e520c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 19/41] pipeline: introduce SWX or instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu ` (21 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index fe44e520c..88d1b2d1a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 20/41] pipeline: introduce SWX xor instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu ` (20 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 88d1b2d1a..6024c800c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 21/41] pipeline: introduce SWX shl instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu ` (19 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6024c800c..419b676bd 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 22/41] pipeline: introduce SWX shr instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu ` (18 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 419b676bd..2098f44c1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 23/41] pipeline: introduce SWX table instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu ` (17 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2098f44c1..887668481 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 24/41] pipeline: introduce SWX extern instruction 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu ` (16 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance in the maibox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 887668481..aaf2aafa5 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 25/41] pipeline: introduce SWX jmp and return instructions 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 26/41] pipeline: add SWX instruction description Cristian Dumitrescu ` (15 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index aaf2aafa5..ef388fec1 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 26/41] pipeline: add SWX instruction description 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu ` (14 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Added SWX instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..d6c086e27 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 27/41] pipeline: add SWX instruction verifier 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 26/41] pipeline: add SWX instruction description Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu ` (13 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Instruction verifier. Executes at instruction translation time during SWX pipeline build, i.e. at initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index ef388fec1..d51fec821 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 28/41] pipeline: add SWX instruction optimizer 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu ` (12 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d51fec821..77eae1927 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 29/41] pipeline: add SWX pipeline query API 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu ` (11 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Query API to be used by the control plane to detect the configuration and state of the SWX pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..bdcc24cee 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup opeation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 77eae1927..da69bab49 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 30/41] pipeline: add SWX pipeline flush 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu ` (10 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Flush the packets currently buffered by the SWX pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index da69bab49..8b7ff56f6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d6c086e27..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -656,6 +656,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 31/41] pipeline: add SWX table update high level API 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu ` (9 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev High-level transaction-oriented API for SWX pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 5 files changed, 1737 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index d214b1aeb..d4a5f77f7 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -22,6 +22,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_ctl.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..44da678c1 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wilcard and LPM match tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index bdcc24cee..786adedb2 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 32/41] pipeline: add SWX pipeline specification file 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 33/41] port: add ethernet device SWX port Cristian Dumitrescu ` (8 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add support for building the SWX pipeline based on specification file with syntax aligned to the P4 language. The specification file may be generated by the P4C compiler in the future. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 1 + lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.h | 26 + lib/librte_pipeline/rte_swx_pipeline_spec.c | 1439 ++++++++++++++++++ 5 files changed, 1468 insertions(+) create mode 100644 lib/librte_pipeline/rte_swx_pipeline_spec.c diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index d4a5f77f7..8f0058fb7 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -22,6 +22,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline_spec.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_ctl.c # install includes diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index be1d9c3a4..65c1a8d6a 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -5,6 +5,7 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', 'rte_swx_pipeline.c', + 'rte_swx_pipeline_spec.c', 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index ec38f0eef..2cb50d571 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -72,6 +72,7 @@ EXPERIMENTAL { rte_swx_pipeline_table_config; rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; + rte_swx_pipeline_build_from_spec; rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 6da5710af..6928e78b6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -643,6 +643,32 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline build from specification file + * + * @param[in] p + * Pipeline handle. + * @param[in] spec + * Pipeline specification file. + * @param[out] err_line + * In case of error and non-NULL, the line number within the *spec* file where + * the error occurred. The first line number in the file is 1. + * @param[out] err_msg + * In case of error and non-NULL, the error message. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Resource with the same name already exists; + * -ENODEV: Extern object or table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg); + /** * Pipeline run * diff --git a/lib/librte_pipeline/rte_swx_pipeline_spec.c b/lib/librte_pipeline/rte_swx_pipeline_spec.c new file mode 100644 index 000000000..d72badd03 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_pipeline_spec.c @@ -0,0 +1,1439 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <errno.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" + +#define MAX_LINE_LENGTH 256 +#define MAX_TOKENS 16 +#define MAX_INSTRUCTION_LENGTH 256 + +#define STRUCT_BLOCK 0 +#define ACTION_BLOCK 1 +#define TABLE_BLOCK 2 +#define TABLE_KEY_BLOCK 3 +#define TABLE_ACTIONS_BLOCK 4 +#define APPLY_BLOCK 5 + +/* + * extobj. + * + * extobj OBJ_NAME instanceof OBJ_TYPE [ pragma OBJ_CREATE_ARGS ] + */ +struct extobj_spec { + char *name; + char *extern_type_name; + char *pragma; +}; + +static void +extobj_spec_free(struct extobj_spec *s) +{ + free(s->name); + free(s->extern_type_name); + free(s->pragma); +} + +static int +extobj_statement_parse(struct extobj_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 4) && (n_tokens != 6)) || + ((n_tokens == 4) && strcmp(tokens[2], "instanceof")) || + ((n_tokens == 6) && (strcmp(tokens[2], "instanceof") || + strcmp(tokens[4], "pragma")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid extobj statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->extern_type_name = strdup(tokens[3]); + s->pragma = (n_tokens == 6) ? strdup(tokens[5]) : NULL; + + if (!s->name || + !s->extern_type_name || + ((n_tokens == 6) && !s->pragma)) { + free(s->name); + free(s->extern_type_name); + free(s->pragma); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * struct. + * + * struct STRUCT_TYPE_NAME { + * bit<SIZE> FIELD_NAME + * ... + * } + */ +struct struct_spec { + char *name; + struct rte_swx_field_params *fields; + uint32_t n_fields; +}; + +static void +struct_spec_free(struct struct_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->n_fields; i++) { + uintptr_t name = (uintptr_t)s->fields[i].name; + + free((void *)name); + } + + free(s->fields); + s->fields = NULL; + + s->n_fields = 0; +} + +static int +struct_statement_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << STRUCT_BLOCK; + + return 0; +} + +static int +struct_block_parse(struct struct_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_field_params *new_fields; + char *p = tokens[0], *name; + uint32_t n_bits; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << STRUCT_BLOCK); + return 0; + } + + /* Check format. */ + if ((n_tokens != 2) || + (strlen(p) < 6) || + (p[0] != 'b') || + (p[1] != 'i') || + (p[2] != 't') || + (p[3] != '<') || + (p[strlen(p) - 1] != '>')) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field statement."; + return -EINVAL; + } + + /* Remove the "bit<" and ">". */ + p[strlen(p) - 1] = 0; + p += 4; + + n_bits = strtoul(p, &p, 0); + if ((p[0]) || + !n_bits || + (n_bits % 8) || + (n_bits > 64)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid struct field size."; + return -EINVAL; + } + + /* spec. */ + name = strdup(tokens[1]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->fields, + s->n_fields + 1, + sizeof(struct rte_swx_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->fields = new_fields; + s->fields[s->n_fields].name = name; + s->fields[s->n_fields].n_bits = n_bits; + s->n_fields++; + + return 0; +} + +/* + * header. + * + * header HEADER_NAME instanceof STRUCT_TYPE_NAME + */ +struct header_spec { + char *name; + char *struct_type_name; +}; + +static void +header_spec_free(struct header_spec *s) +{ + free(s->name); + free(s->struct_type_name); +} + +static int +header_statement_parse(struct header_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 4) || strcmp(tokens[2], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid header statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->struct_type_name = strdup(tokens[3]); + + if (!s->name || !s->struct_type_name) { + free(s->name); + free(s->struct_type_name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * metadata. + * + * metadata instanceof STRUCT_TYPE_NAME + */ +struct metadata_spec { + char *struct_type_name; +}; + +static void +metadata_spec_free(struct metadata_spec *s) +{ + free(s->struct_type_name); +} + +static int +metadata_statement_parse(struct metadata_spec *s, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[1], "instanceof")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid metadata statement."; + return -EINVAL; + } + + /* spec. */ + s->struct_type_name = strdup(tokens[2]); + if (!s->struct_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; +} + +/* + * action. + * + * action ACTION_NAME args none | instanceof STRUCT_TYPE_NAME { + * INSTRUCTION + * ... + * } + */ +struct action_spec { + char *name; + char *args_struct_type_name; + const char **instructions; + uint32_t n_instructions; +}; + +static void +action_spec_free(struct action_spec *s) +{ + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + free(s->args_struct_type_name); + s->args_struct_type_name = NULL; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +action_statement_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if (((n_tokens != 5) && (n_tokens != 6)) || + ((n_tokens == 5) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + strcmp(tokens[4], "{"))) || + ((n_tokens == 6) && + (strcmp(tokens[2], "args") || + strcmp(tokens[3], "instanceof") || + strcmp(tokens[5], "{")))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + s->args_struct_type_name = (n_tokens == 6) ? strdup(tokens[4]) : NULL; + + if ((!s->name) || ((n_tokens == 6) && !s->args_struct_type_name)) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << ACTION_BLOCK; + + return 0; +} + +static int +action_block_parse(struct action_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << ACTION_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * table. + * + * table { + * key { + * MATCH_FIELD_NAME exact | wildcard | lpm + * ... + * } + * actions { + * ACTION_NAME + * ... + * } + * default_action ACTION_NAME args none | ARGS_BYTE_ARRAY [ const ] + * instanceof TABLE_TYPE_NAME + * pragma ARGS + * size SIZE + * } + */ +struct table_spec { + char *name; + struct rte_swx_pipeline_table_params params; + char *recommended_table_type_name; + char *args; + uint32_t size; +}; + +static void +table_spec_free(struct table_spec *s) +{ + uintptr_t default_action_name; + uint32_t i; + + if (!s) + return; + + free(s->name); + s->name = NULL; + + for (i = 0; i < s->params.n_fields; i++) { + uintptr_t name = (uintptr_t)s->params.fields[i].name; + + free((void *)name); + } + + free(s->params.fields); + s->params.fields = NULL; + + s->params.n_fields = 0; + + for (i = 0; i < s->params.n_actions; i++) { + uintptr_t name = (uintptr_t)s->params.action_names[i]; + + free((void *)name); + } + + free(s->params.action_names); + s->params.action_names = NULL; + + s->params.n_actions = 0; + + default_action_name = (uintptr_t)s->params.default_action_name; + free((void *)default_action_name); + s->params.default_action_name = NULL; + + free(s->params.default_action_data); + s->params.default_action_data = NULL; + + s->params.default_action_is_const = 0; + + free(s->recommended_table_type_name); + s->recommended_table_type_name = NULL; + + free(s->args); + s->args = NULL; + + s->size = 0; +} + +static int +table_key_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid key statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_KEY_BLOCK; + + return 0; +} + +static int +table_key_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + struct rte_swx_match_field_params *new_fields; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_KEY_BLOCK); + return 0; + } + + /* Check input arguments. */ + if ((n_tokens != 2) || + (strcmp(tokens[1], "exact") && + strcmp(tokens[1], "wildcard") && + strcmp(tokens[1], "lpm"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid match field statement."; + return -EINVAL; + } + + if (!strcmp(tokens[1], "wildcard")) + match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + if (!strcmp(tokens[1], "lpm")) + match_type = RTE_SWX_TABLE_MATCH_LPM; + if (!strcmp(tokens[1], "exact")) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_fields = reallocarray(s->params.fields, + s->params.n_fields + 1, + sizeof(struct rte_swx_match_field_params)); + if (!new_fields) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.fields = new_fields; + s->params.fields[s->params.n_fields].name = name; + s->params.fields[s->params.n_fields].match_type = match_type; + s->params.n_fields++; + + return 0; +} + +static int +table_actions_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid actions statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_ACTIONS_BLOCK; + + return 0; +} + +static int +table_actions_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + const char **new_action_names; + char *name; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_ACTIONS_BLOCK); + return 0; + } + + /* Check input arguments. */ + if (n_tokens != 1) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid action name statement."; + return -EINVAL; + } + + name = strdup(tokens[0]); + if (!name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_action_names = reallocarray(s->params.action_names, + s->params.n_actions + 1, + sizeof(char *)); + if (!new_action_names) { + free(name); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->params.action_names = new_action_names; + s->params.action_names[s->params.n_actions] = name; + s->params.n_actions++; + + return 0; +} + +static int +table_statement_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 3) || strcmp(tokens[2], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid table statement."; + return -EINVAL; + } + + /* spec. */ + s->name = strdup(tokens[1]); + if (!s->name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + /* block_mask. */ + *block_mask |= 1 << TABLE_BLOCK; + + return 0; +} + +static int +table_block_parse(struct table_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + if (*block_mask & (1 << TABLE_KEY_BLOCK)) + return table_key_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (*block_mask & (1 << TABLE_ACTIONS_BLOCK)) + return table_actions_block_parse(s, + block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << TABLE_BLOCK); + return 0; + } + + if (!strcmp(tokens[0], "key")) + return table_key_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "actions")) + return table_actions_statement_parse(block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + + if (!strcmp(tokens[0], "default_action")) { + if (((n_tokens != 4) && (n_tokens != 5)) || + strcmp(tokens[2], "args") || + strcmp(tokens[3], "none") || + ((n_tokens == 5) && strcmp(tokens[4], "const"))) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid default_action statement."; + return -EINVAL; + } + + if (s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate default_action stmt."; + return -EINVAL; + } + + s->params.default_action_name = strdup(tokens[1]); + if (!s->params.default_action_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + if (n_tokens == 5) + s->params.default_action_is_const = 1; + + return 0; + } + + if (!strcmp(tokens[0], "instanceof")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid instanceof statement."; + return -EINVAL; + } + + if (s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate instanceof statement."; + return -EINVAL; + } + + s->recommended_table_type_name = strdup(tokens[1]); + if (!s->recommended_table_type_name) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "pragma")) { + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + if (s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Duplicate pragma statement."; + return -EINVAL; + } + + s->args = strdup(tokens[1]); + if (!s->args) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + return 0; + } + + if (!strcmp(tokens[0], "size")) { + char *p = tokens[1]; + + if (n_tokens != 2) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid pragma statement."; + return -EINVAL; + } + + s->size = strtoul(p, &p, 0); + if (p[0]) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid size argument."; + return -EINVAL; + } + + return 0; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid statement."; + return -EINVAL; +} + +/* + * apply. + * + * apply { + * INSTRUCTION + * ... + * } + */ +struct apply_spec { + const char **instructions; + uint32_t n_instructions; +}; + +static void +apply_spec_free(struct apply_spec *s) +{ + uint32_t i; + + if (!s) + return; + + for (i = 0; i < s->n_instructions; i++) { + uintptr_t instr = (uintptr_t)s->instructions[i]; + + free((void *)instr); + } + + free(s->instructions); + s->instructions = NULL; + + s->n_instructions = 0; +} + +static int +apply_statement_parse(uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + /* Check format. */ + if ((n_tokens != 2) || strcmp(tokens[1], "{")) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Invalid apply statement."; + return -EINVAL; + } + + /* block_mask. */ + *block_mask |= 1 << APPLY_BLOCK; + + return 0; +} + +static int +apply_block_parse(struct apply_spec *s, + uint32_t *block_mask, + char **tokens, + uint32_t n_tokens, + uint32_t n_lines, + uint32_t *err_line, + const char **err_msg) +{ + char buffer[MAX_INSTRUCTION_LENGTH], *instr; + const char **new_instructions; + uint32_t i; + + /* Handle end of block. */ + if ((n_tokens == 1) && !strcmp(tokens[0], "}")) { + *block_mask &= ~(1 << APPLY_BLOCK); + return 0; + } + + /* spec. */ + buffer[0] = 0; + for (i = 0; i < n_tokens; i++) { + if (i) + strcat(buffer, " "); + strcat(buffer, tokens[i]); + } + + instr = strdup(buffer); + if (!instr) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + new_instructions = reallocarray(s->instructions, + s->n_instructions + 1, + sizeof(char *)); + if (!new_instructions) { + free(instr); + + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Memory allocation failed."; + return -ENOMEM; + } + + s->instructions = new_instructions; + s->instructions[s->n_instructions] = instr; + s->n_instructions++; + + return 0; +} + +/* + * Pipeline. + */ +int +rte_swx_pipeline_build_from_spec(struct rte_swx_pipeline *p, + FILE *spec, + uint32_t *err_line, + const char **err_msg) +{ + struct extobj_spec extobj_spec = {0}; + struct struct_spec struct_spec = {0}; + struct header_spec header_spec = {0}; + struct metadata_spec metadata_spec = {0}; + struct action_spec action_spec = {0}; + struct table_spec table_spec = {0}; + struct apply_spec apply_spec = {0}; + uint32_t n_lines; + uint32_t block_mask = 0; + int status; + + /* Check the input arguments. */ + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null pipeline arument."; + status = -EINVAL; + goto error; + } + + if (!p) { + if (err_line) + *err_line = 0; + if (err_msg) + *err_msg = "Null specification file argument."; + status = -EINVAL; + goto error; + } + + for (n_lines = 1; ; n_lines++) { + char line[MAX_LINE_LENGTH]; + char *tokens[MAX_TOKENS], *ptr = line; + uint32_t n_tokens = 0; + + /* Read next line. */ + if (!fgets(line, sizeof(line), spec)) + break; + + /* Parse the line into tokens. */ + for ( ; ; ) { + char *token; + + /* Get token. */ + token = strtok_r(ptr, " \f\n\r\t\v", &ptr); + if (!token) + break; + + /* Handle comments. */ + if ((token[0] == '#') || + (token[0] == ';') || + ((token[0] == '/') && (token[1] == '/'))) { + break; + } + + /* Handle excessively long lines. */ + if (n_tokens >= MAX_TOKENS) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Too many tokens."; + status = -EINVAL; + goto error; + } + + /* Save token. */ + tokens[n_tokens] = token; + n_tokens++; + } + + /* Handle empty lines. */ + if (!n_tokens) + continue; + + /* struct block. */ + if (block_mask & (1 << STRUCT_BLOCK)) { + status = struct_block_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << STRUCT_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_struct_type_register(p, + struct_spec.name, + struct_spec.fields, + struct_spec.n_fields); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Struct registration error."; + goto error; + } + + struct_spec_free(&struct_spec); + + continue; + } + + /* action block. */ + if (block_mask & (1 << ACTION_BLOCK)) { + status = action_block_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << ACTION_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_action_config(p, + action_spec.name, + action_spec.args_struct_type_name, + action_spec.instructions, + action_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Action config error."; + goto error; + } + + action_spec_free(&action_spec); + + continue; + } + + /* table block. */ + if (block_mask & (1 << TABLE_BLOCK)) { + status = table_block_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << TABLE_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_table_config(p, + table_spec.name, + &table_spec.params, + table_spec.recommended_table_type_name, + table_spec.args, + table_spec.size); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Table configuration error."; + goto error; + } + + table_spec_free(&table_spec); + + continue; + } + + /* apply block. */ + if (block_mask & (1 << APPLY_BLOCK)) { + status = apply_block_parse(&apply_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + if (block_mask & (1 << APPLY_BLOCK)) + continue; + + /* End of block. */ + status = rte_swx_pipeline_instructions_config(p, + apply_spec.instructions, + apply_spec.n_instructions); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline instructions err."; + goto error; + } + + apply_spec_free(&apply_spec); + + continue; + } + + /* extobj. */ + if (!strcmp(tokens[0], "extobj")) { + status = extobj_statement_parse(&extobj_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_extern_object_config(p, + extobj_spec.name, + extobj_spec.extern_type_name, + extobj_spec.pragma); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Extern object config err."; + goto error; + } + + extobj_spec_free(&extobj_spec); + + continue; + } + + /* struct. */ + if (!strcmp(tokens[0], "struct")) { + status = struct_statement_parse(&struct_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* header. */ + if (!strcmp(tokens[0], "header")) { + status = header_statement_parse(&header_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_header_register(p, + header_spec.name, + header_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Header registration error."; + goto error; + } + + header_spec_free(&header_spec); + + continue; + } + + /* metadata. */ + if (!strcmp(tokens[0], "metadata")) { + status = metadata_statement_parse(&metadata_spec, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + status = rte_swx_pipeline_packet_metadata_register(p, + metadata_spec.struct_type_name); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Meta-data reg err."; + goto error; + } + + metadata_spec_free(&metadata_spec); + + continue; + } + + /* action. */ + if (!strcmp(tokens[0], "action")) { + status = action_statement_parse(&action_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* table. */ + if (!strcmp(tokens[0], "table")) { + status = table_statement_parse(&table_spec, + &block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* apply. */ + if (!strcmp(tokens[0], "apply")) { + status = apply_statement_parse(&block_mask, + tokens, + n_tokens, + n_lines, + err_line, + err_msg); + if (status) + goto error; + + continue; + } + + /* Anything else. */ + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Unknown statement."; + status = -EINVAL; + goto error; + } + + /* Handle unfinished block. */ + if (block_mask) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Missing }."; + status = -EINVAL; + goto error; + } + + /* Pipeline build. */ + status = rte_swx_pipeline_build(p); + if (status) { + if (err_line) + *err_line = n_lines; + if (err_msg) + *err_msg = "Pipeline build error."; + goto error; + } + + return 0; + +error: + extobj_spec_free(&extobj_spec); + struct_spec_free(&struct_spec); + header_spec_free(&header_spec); + metadata_spec_free(&metadata_spec); + action_spec_free(&action_spec); + table_spec_free(&table_spec); + apply_spec_free(&apply_spec); + return status; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 33/41] port: add ethernet device SWX port 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 34/41] port: add source and sink SWX ports Cristian Dumitrescu ` (7 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add the Ethernet device input/output port type for the SWX pipeline. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 5 files changed, 375 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 4221618b3..682336c01 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -38,6 +38,7 @@ endif SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -56,5 +57,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 34/41] port: add source and sink SWX ports 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 33/41] port: add ethernet device SWX port Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 35/41] table: add exact match SWX table Cristian Dumitrescu ` (6 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add the PCAP file-based source (input) and sink (output) port types for the SWX pipeline. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 5 files changed, 400 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 682336c01..e4a9865d9 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -39,6 +39,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_source_sink.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -58,5 +59,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_source_sink.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..4180cba1c --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_common.h> +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +struct sink { + struct rte_swx_port_out_stats stats; + +#ifdef RTE_PORT_PCAP + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +#endif +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); +#endif + + free(p); +} + +static void * +sink_create(void *args __rte_unused) +{ + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + +#ifdef RTE_PORT_PCAP + if (args) { + struct rte_swx_port_sink_params *params = args; + + if (params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, + params->file_name); + if (!p->f_dump) + goto error; + } + } +#endif + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + +#ifdef RTE_PORT_PCAP + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } +#endif + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 35/41] table: add exact match SWX table 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 34/41] port: add source and sink SWX ports Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 36/41] examples/pipeline: add new example application Cristian Dumitrescu ` (5 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add the exact match table type for the SWX pipeline. Used under the hood by the SWX pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/Makefile | 2 + lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 5 files changed, 894 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 9df58698d..f544fd5af 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -34,6 +34,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_ext.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_lru.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_array.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_stub.c +SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_swx_table_em.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table.h @@ -56,5 +57,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table_em.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 36/41] examples/pipeline: add new example application 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 35/41] table: add exact match SWX table Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (4 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add new example application to showcase the API of the newly introduced SWX pipeline type. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/Makefile | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 80 +++++ examples/pipeline/main.c | 52 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1328 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/examples/Makefile b/examples/Makefile index b7e99a2f7..b1ebac681 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -61,6 +61,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += packet_ordering ifeq ($(CONFIG_RTE_ARCH_X86_64),y) DIRS-y += performance-thread endif +DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline DIRS-$(CONFIG_RTE_LIBRTE_IEEE1588) += ptpclient DIRS-$(CONFIG_RTE_LIBRTE_METER) += qos_meter DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += qos_sched diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..3c85e9e40 --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,80 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + +else + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, detect a build directory, by looking for a path with a .config +RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config))))) + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) +$(info This application can only operate in a linux environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +clean: +else + +INC += $(sort $(wildcard *.h)) + +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := $(SRCS-y) + +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(SRCDIR) +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif +endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..353e62c10 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + /* Dispatch loop */ + for ( ; ; ); +} + diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..688870f97 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_table_em.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p = NULL; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + goto error; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) + goto error; + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) + goto error; + +#ifdef RTE_PORT_PCAP + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) + goto error; +#endif + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) + goto error; + + status = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + if (status) + goto error; + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) + goto error; + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; + +error: + rte_swx_pipeline_free(p); + return NULL; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..0d1474c55 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Master thead: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Master thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Master thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Master thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Master thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 37/41] examples/pipeline: add message passing mechanism 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 36/41] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu ` (3 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 136 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 517 insertions(+), 2 deletions(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 3c85e9e40..eb1085f7c 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 353e62c10..1a63c1237 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,7 +167,18 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; /* Dispatch loop */ - for ( ; ; ); -} + for ( ; ; ) { + conn_poll_for_conn(conn); + conn_poll_for_msg(conn); + } +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 38/41] examples/pipeline: add configuration commands 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (2 subsequent siblings) 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1400 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 11 +- examples/pipeline/meson.build | 1 + 5 files changed, 1431 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index eb1085f7c..dcc0f67bf 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..7a1863ee7 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1400 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <spec_file>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p = NULL; + FILE *spec = NULL; + uint32_t err_line; + const char *err_msg; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + spec = fopen(tokens[3], "r"); + if (!spec) { + snprintf(out, out_size, "Cannot open file %s.\n", tokens[3]); + return; + } + + status = rte_swx_pipeline_build_from_spec(p->p, + spec, + &err_line, + &err_msg); + fclose(spec); + if (status) { + snprintf(out, out_size, "Error %d at line %u: %s\n.", + status, err_line, err_msg); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 + " empty %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, "\tPort %u:" + " packets %" PRIu64 + " bytes %" PRIu64 "\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + struct pipeline *p; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + p = pipeline_find(obj, pipeline_name); + if (!p || !p->ctl) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 1a63c1237..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); @@ -175,6 +183,7 @@ main(int argc, char **argv) status); return status; }; + /* Dispatch loop */ for ( ; ; ) { conn_poll_for_conn(conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 39/41] examples/pipeline: add l2fwd example 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add L2 Forwarding example to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd.cli | 25 ++++++ examples/pipeline/examples/l2fwd.spec | 42 +++++++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 +++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd.spec create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..d2ac092d9 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd.spec b/examples/pipeline/examples/l2fwd.spec new file mode 100644 index 000000000..0aebafd07 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.spec @@ -0,0 +1,42 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Meta-data. +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions. +// +action NoAction args none { + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + NoAction + } + + default_action NoAction args none const +} + +// +// Pipeline. +// +apply { + rx m.port_in + table stub + tx m.port_in +} diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..be7773b58 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 40/41] examples/pipeline: add l2fwd with MAC swap example 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add L2 Forwarding example with MAC destination and source address swap to the SWX pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/l2fwd_macswp.cli | 25 ++++++++ examples/pipeline/examples/l2fwd_macswp.spec | 59 +++++++++++++++++++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++++++ 3 files changed, 104 insertions(+) create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp.spec create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..5a4b95a35 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp.spec b/examples/pipeline/examples/l2fwd_macswp.spec new file mode 100644 index 000000000..e81f20622 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.spec @@ -0,0 +1,59 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +// +// Packet headers. +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ether_type +} + +header ethernet instanceof ethernet_h + +// +// Packet meta-data. +// +struct metadata_t { + bit<32> port + bit<48> addr +} + +metadata instanceof metadata_t + +// +// Actions. +// +action macswp args none { + mov m.addr h.ethernet.dst_addr + mov h.ethernet.dst_addr h.ethernet.src_addr + mov h.ethernet.src_addr m.addr + return +} + +// +// Tables. +// +table stub { + key { + } + + actions { + macswp + } + + default_action macswp args none const +} + +// +// Pipeline. +// +apply { + rx m.port + extract h.ethernet + table stub + xor m.port 1 + emit h.ethernet + tx m.port +} diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..9044d7d7f --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build ./examples/l2fwd_macswp.spec + +thread 1 pipeline PIPELINE0 enable -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH v2 41/41] examples/pipeline: add VXLAN encapsulation example 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu ` (39 preceding siblings ...) 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-09-07 21:40 ` Cristian Dumitrescu 40 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-09-07 21:40 UTC (permalink / raw) To: dev Add VXLAN encapsulation example to the SWX pipeline application. The VXLAN tunnels can be generated with the vxlan_table.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/examples/vxlan.cli | 27 ++++ examples/pipeline/examples/vxlan.spec | 173 +++++++++++++++++++++ examples/pipeline/examples/vxlan_pcap.cli | 22 +++ examples/pipeline/examples/vxlan_table.py | 71 +++++++++ examples/pipeline/examples/vxlan_table.txt | 16 ++ 5 files changed, 309 insertions(+) create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.spec create mode 100644 examples/pipeline/examples/vxlan_pcap.cli create mode 100644 examples/pipeline/examples/vxlan_table.py create mode 100644 examples/pipeline/examples/vxlan_table.txt diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..7648f25d9 --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.spec b/examples/pipeline/examples/vxlan.spec new file mode 100644 index 000000000..47106471f --- /dev/null +++ b/examples/pipeline/examples/vxlan.spec @@ -0,0 +1,173 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +// +// Headers +// +struct ethernet_h { + bit<48> dst_addr + bit<48> src_addr + bit<16> ethertype +} + +struct ipv4_h { + bit<8> ver_ihl + bit<8> diffserv + bit<16> total_len + bit<16> identification + bit<16> flags_offset + bit<8> ttl + bit<8> protocol + bit<16> hdr_checksum + bit<32> src_addr + bit<32> dst_addr +} + +struct udp_h { + bit<16> src_port + bit<16> dst_port + bit<16> length + bit<16> checksum +} + +struct vxlan_h { + bit<8> flags + bit<24> reserved + bit<24> vni + bit<8> reserved2 +} + +header ethernet instanceof ethernet_h +header ipv4 instanceof ipv4_h +header outer_ethernet instanceof ethernet_h +header outer_ipv4 instanceof ipv4_h +header outer_udp instanceof udp_h +header outer_vxlan instanceof vxlan_h + +// +// Meta-data +// +struct metadata_t { + bit<32> port_in + bit<32> port_out +} + +metadata instanceof metadata_t + +// +// Actions +// +struct vxlan_encap_args_t { + bit<48> ethernet_dst_addr + bit<48> ethernet_src_addr + bit<16> ethernet_ether_type + bit<8> ipv4_ver_ihl + bit<8> ipv4_diffserv + bit<16> ipv4_total_len + bit<16> ipv4_identification + bit<16> ipv4_flags_offset + bit<8> ipv4_ttl + bit<8> ipv4_protocol + bit<16> ipv4_hdr_checksum + bit<32> ipv4_src_addr + bit<32> ipv4_dst_addr + bit<16> udp_src_port + bit<16> udp_dst_port + bit<16> udp_length + bit<16> udp_checksum + bit<8> vxlan_flags + bit<24> vxlan_reserved + bit<24> vxlan_vni + bit<8> vxlan_reserved2 + bit<32> port_out +} + +// Input frame: +// Ethernet (14) | IPv4 (total_len) +// +// Output frame: +// Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | Ethernet FCS (4) +// +// Note: The input frame has its FCS removed before encapsulation in the output +// frame. +// +// Assumption: When read from the table, the outer IPv4 and UDP headers contain +// the following fields: +// - t.ipv4_total_len: Set to 50, which covers the length of: +// - The outer IPv4 header (20 bytes); +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.ipv4_hdr_checksum: Includes the above total length. +// - t.udp_length: Set to 30, which covers the length of: +// - The outer UDP header (8 bytes); +// - The outer VXLAN header (8 bytes); +// - The inner Ethernet header (14 bytes); +// - t.udp_checksum: Set to 0. +// +// Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, +// the outer IPv4 and UDP headers are updated as follows: +// - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len +// - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len +// - h.outer_udp.length = t.udp_length + h.ipv4.total_len +// - h.outer_udp.checksum: No change. +// + +action vxlan_encap args instanceof vxlan_encap_args_t { + //Copy from table entry to haders and metadata. + dma h.outer_ethernet t.ethernet_dst_addr + dma h.outer_ipv4 t.ipv4_ver_ihl + dma h.outer_udp t.udp_src_port + dma h.outer_vxlan t.vxlan_flags + mov m.port_out t.port_out + + //Update h.outer_ipv4.total_len field. + add h.outer_ipv4.total_len h.ipv4.total_len + + //Update h.outer_ipv4.hdr_checksum field. + ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len + + //Update h.outer_udp.length field. + add h.outer_udp.length h.ipv4.total_len + + return +} + +action drop args none { + mov m.port_out 4 + tx m.port_out +} + +// +// Tables. +// +table vxlan_table { + key { + h.ethernet.dst_addr exact + } + + actions { + vxlan_encap + drop + } + + default_action drop args none + size 1048576 +} + +// +// Pipeline. +// +apply { + rx m.port_in + extract h.ethernet + extract h.ipv4 + table vxlan_table + emit h.outer_ethernet + emit h.outer_ipv4 + emit h.outer_udp + emit h.outer_vxlan + emit h.ethernet + emit h.ipv4 + tx m.port_out +} diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c6975343e --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build ./examples/vxlan.spec +pipeline PIPELINE0 table vxlan_table update ./examples/vxlan_table.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan_table.py b/examples/pipeline/examples/vxlan_table.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan_table.txt b/examples/pipeline/examples/vxlan_table.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan_table.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 02/40] pipeline: add input port 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 03/40] pipeline: add output port Cristian Dumitrescu ` (37 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add input ports to the pipeline. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The RX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 209 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 54 +++++ lib/librte_port/Makefile | 1 + lib/librte_port/meson.build | 3 +- lib/librte_port/rte_swx_port.h | 118 +++++++++++ 6 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 lib/librte_port/rte_swx_port.h diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 39593f1ee..a9ebd3b1f 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -56,6 +56,8 @@ EXPERIMENTAL { rte_table_action_ttl_read; rte_table_action_crypto_sym_session_get; rte_swx_pipeline_config; + rte_swx_pipeline_port_in_type_register; + rte_swx_pipeline_port_in_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2319d4570..5b1559209 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5,6 +5,7 @@ #include <string.h> #include <stdio.h> #include <errno.h> +#include <sys/queue.h> #include <rte_common.h> @@ -19,14 +20,206 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Input port. + */ +struct port_in_type { + TAILQ_ENTRY(port_in_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_in_ops ops; +}; + +TAILQ_HEAD(port_in_type_tailq, port_in_type); + +struct port_in { + TAILQ_ENTRY(port_in) node; + struct port_in_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_in_tailq, port_in); + +struct port_in_runtime { + rte_swx_port_in_pkt_rx_t pkt_rx; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { + struct port_in_type_tailq port_in_types; + struct port_in_tailq ports_in; + + struct port_in_runtime *in; + + uint32_t n_ports_in; int build_done; int numa_node; }; +/* + * Input port. + */ +static struct port_in_type * +port_in_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_in_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_in_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops) +{ + struct port_in_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_rx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_in_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_in_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_in_types, elem, node); + + return 0; +} + +static struct port_in * +port_in_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_in *port; + + TAILQ_FOREACH(port, &p->ports_in, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_in_type *type = NULL; + struct port_in *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_in_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_in_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_in)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_in, port, node); + if (p->n_ports_in < port_id + 1) + p->n_ports_in = port_id + 1; + + return 0; +} + +static int +port_in_build(struct rte_swx_pipeline *p) +{ + struct port_in *port; + uint32_t i; + + CHECK(p->n_ports_in, EINVAL); + CHECK(rte_is_power_of_2(p->n_ports_in), EINVAL); + + for (i = 0; i < p->n_ports_in; i++) + CHECK(port_in_find(p, i), EINVAL); + + p->in = calloc(p->n_ports_in, sizeof(struct port_in_runtime)); + CHECK(p->in, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_in, node) { + struct port_in_runtime *in = &p->in[port->id]; + + in->pkt_rx = port->type->ops.pkt_rx; + in->obj = port->obj; + } + + return 0; +} + +static void +port_in_build_free(struct rte_swx_pipeline *p) +{ + free(p->in); + p->in = NULL; +} + +static void +port_in_free(struct rte_swx_pipeline *p) +{ + port_in_build_free(p); + + /* Input ports. */ + for ( ; ; ) { + struct port_in *port; + + port = TAILQ_FIRST(&p->ports_in); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_in, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Input port types. */ + for ( ; ; ) { + struct port_in_type *elem; + + elem = TAILQ_FIRST(&p->port_in_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_in_types, elem, node); + free(elem); + } +} /* * Pipeline. @@ -44,6 +237,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->port_in_types); + TAILQ_INIT(&pipeline->ports_in); + pipeline->numa_node = numa_node; *p = pipeline; @@ -56,15 +252,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_in_free(p); + free(p); } int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { + int status; + CHECK(p, EINVAL); CHECK(p->build_done == 0, EEXIST); + status = port_in_build(p); + if (status) + goto error; + p->build_done = 1; return 0; + +error: + port_in_build_free(p); + + return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index ded26a4e4..3dbe7ce0b 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -18,6 +18,12 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" + +/** Name size. */ +#ifndef RTE_SWX_NAME_SIZE +#define RTE_SWX_NAME_SIZE 64 +#endif /* * Pipeline setup and operation */ @@ -43,6 +49,54 @@ int rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node); +/* + * Pipeline input ports + */ + +/** + * Pipeline input port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Input port type name. + * @param[in] ops + * Input port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Input port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_in_ops *ops); + +/** + * Pipeline input port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Input port ID. + * @param[in] port_type_name + * Existing input port type name. + * @param[in] args + * Input port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Input port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); /** * Pipeline build * diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 57d2aedbc..4221618b3 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -55,5 +55,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 0d5ede44a..5b5fbf6c4 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -21,7 +21,8 @@ headers = files( 'rte_port_sched.h', 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', - 'rte_port_eventdev.h') + 'rte_port_eventdev.h', + 'rte_swx_port.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h new file mode 100644 index 000000000..a6f80de9a --- /dev/null +++ b/lib/librte_port/rte_swx_port.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_H__ +#define __INCLUDE_RTE_SWX_PORT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Port + * + * Packet I/O port interface. + */ + +#include <stdint.h> + +/** Packet. */ +struct rte_swx_pkt { + /** Opaque packet handle. */ + void *handle; + + /** Buffer where the packet is stored. */ + uint8_t *pkt; + + /** Packet buffer offset of the first packet byte. */ + uint32_t offset; + + /** Packet length in bytes. */ + uint32_t length; +}; + +/* + * Input port + */ + +/** + * Input port create + * + * @param[in] args + * Arguments for input port creation. Format specific to each port type. + * @return + * Handle to input port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_in_create_t)(void *args); + +/** + * Input port free + * + * @param[in] args + * Input port handle. + */ +typedef void +(*rte_swx_port_in_free_t)(void *port); + +/** + * Input port packet receive + * + * @param[in] port + * Input port handle. + * @param[out] pkt + * Received packet. Only valid when the function returns 1. Must point to + * valid memory. + * @return + * 0 when no packet was received, 1 when a packet was received. No other + * return values are allowed. + */ +typedef int +(*rte_swx_port_in_pkt_rx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** Input port statistics counters. */ +struct rte_swx_port_in_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; + + /** Number of empty polls. */ + uint64_t n_empty; +}; + +/** + * Input port statistics counters read + * + * @param[in] port + * Input port handle. + * @param[out] stats + * Input port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_in_stats_read_t)(void *port, + struct rte_swx_port_in_stats *stats); + +/** Input port operations. */ +struct rte_swx_port_in_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_in_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_in_free_t free; + + /** Packet reception. Must be non-NULL. */ + rte_swx_port_in_pkt_rx_t pkt_rx; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_in_stats_read_t stats_read; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 03/40] pipeline: add output port 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 02/40] pipeline: add input port Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 04/40] pipeline: add headers and meta-data Cristian Dumitrescu ` (36 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add output ports to the pipeline. Each port instantiates a port type that defines the port operations, e.g. ethdev port, PCAP port, etc. The TX interface is single packet, with packet batching internally for performance. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 2 + lib/librte_pipeline/rte_swx_pipeline.c | 200 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 50 +++++ lib/librte_port/rte_swx_port.h | 84 ++++++++ 4 files changed, 336 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index a9ebd3b1f..88fd38ca8 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -58,6 +58,8 @@ EXPERIMENTAL { rte_swx_pipeline_config; rte_swx_pipeline_port_in_type_register; rte_swx_pipeline_port_in_config; + rte_swx_pipeline_port_out_type_register; + rte_swx_pipeline_port_out_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 5b1559209..7aeac8cc8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -45,16 +45,46 @@ struct port_in_runtime { void *obj; }; +/* + * Output port. + */ +struct port_out_type { + TAILQ_ENTRY(port_out_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct rte_swx_port_out_ops ops; +}; + +TAILQ_HEAD(port_out_type_tailq, port_out_type); + +struct port_out { + TAILQ_ENTRY(port_out) node; + struct port_out_type *type; + void *obj; + uint32_t id; +}; + +TAILQ_HEAD(port_out_tailq, port_out); + +struct port_out_runtime { + rte_swx_port_out_pkt_tx_t pkt_tx; + rte_swx_port_out_flush_t flush; + void *obj; +}; + /* * Pipeline. */ struct rte_swx_pipeline { struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; + struct port_out_type_tailq port_out_types; + struct port_out_tailq ports_out; struct port_in_runtime *in; + struct port_out_runtime *out; uint32_t n_ports_in; + uint32_t n_ports_out; int build_done; int numa_node; }; @@ -221,6 +251,168 @@ port_in_free(struct rte_swx_pipeline *p) } } +/* + * Output port. + */ +static struct port_out_type * +port_out_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct port_out_type *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->port_out_types, node) + if (!strcmp(elem->name, name)) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops) +{ + struct port_out_type *elem; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->free, EINVAL); + CHECK(ops->pkt_tx, EINVAL); + CHECK(ops->stats_read, EINVAL); + + CHECK(!port_out_type_find(p, name), EEXIST); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct port_out_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->port_out_types, elem, node); + + return 0; +} + +static struct port_out * +port_out_find(struct rte_swx_pipeline *p, uint32_t port_id) +{ + struct port_out *port; + + TAILQ_FOREACH(port, &p->ports_out, node) + if (port->id == port_id) + return port; + + return NULL; +} + +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args) +{ + struct port_out_type *type = NULL; + struct port_out *port = NULL; + void *obj = NULL; + + CHECK(p, EINVAL); + + CHECK(!port_out_find(p, port_id), EINVAL); + + CHECK_NAME(port_type_name, EINVAL); + type = port_out_type_find(p, port_type_name); + CHECK(type, EINVAL); + + obj = type->ops.create(args); + CHECK(obj, ENODEV); + + /* Node allocation. */ + port = calloc(1, sizeof(struct port_out)); + CHECK(port, ENOMEM); + + /* Node initialization. */ + port->type = type; + port->obj = obj; + port->id = port_id; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->ports_out, port, node); + if (p->n_ports_out < port_id + 1) + p->n_ports_out = port_id + 1; + + return 0; +} + +static int +port_out_build(struct rte_swx_pipeline *p) +{ + struct port_out *port; + uint32_t i; + + CHECK(p->n_ports_out, EINVAL); + + for (i = 0; i < p->n_ports_out; i++) + CHECK(port_out_find(p, i), EINVAL); + + p->out = calloc(p->n_ports_out, sizeof(struct port_out_runtime)); + CHECK(p->out, ENOMEM); + + TAILQ_FOREACH(port, &p->ports_out, node) { + struct port_out_runtime *out = &p->out[port->id]; + + out->pkt_tx = port->type->ops.pkt_tx; + out->flush = port->type->ops.flush; + out->obj = port->obj; + } + + return 0; +} + +static void +port_out_build_free(struct rte_swx_pipeline *p) +{ + free(p->out); + p->out = NULL; +} + +static void +port_out_free(struct rte_swx_pipeline *p) +{ + port_out_build_free(p); + + /* Output ports. */ + for ( ; ; ) { + struct port_out *port; + + port = TAILQ_FIRST(&p->ports_out); + if (!port) + break; + + TAILQ_REMOVE(&p->ports_out, port, node); + port->type->ops.free(port->obj); + free(port); + } + + /* Output port types. */ + for ( ; ; ) { + struct port_out_type *elem; + + elem = TAILQ_FIRST(&p->port_out_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->port_out_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -239,6 +431,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) /* Initialization. */ TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); + TAILQ_INIT(&pipeline->port_out_types); + TAILQ_INIT(&pipeline->ports_out); pipeline->numa_node = numa_node; @@ -252,6 +446,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + port_out_free(p); port_in_free(p); free(p); @@ -269,10 +464,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = port_out_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + port_out_build_free(p); port_in_build_free(p); return status; diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 3dbe7ce0b..2be83bd35 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -97,6 +97,56 @@ rte_swx_pipeline_port_in_config(struct rte_swx_pipeline *p, uint32_t port_id, const char *port_type_name, void *args); + +/* + * Pipeline output ports + */ + +/** + * Pipeline output port type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Output port type name. + * @param[in] ops + * Output port type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Output port type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_port_out_ops *ops); + +/** + * Pipeline output port configure + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Output port ID. + * @param[in] port_type_name + * Existing output port type name. + * @param[in] args + * Output port creation arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -ENODEV: Output port object creation error. + */ +__rte_experimental +int +rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, + uint32_t port_id, + const char *port_type_name, + void *args); + /** * Pipeline build * diff --git a/lib/librte_port/rte_swx_port.h b/lib/librte_port/rte_swx_port.h index a6f80de9a..4beb59991 100644 --- a/lib/librte_port/rte_swx_port.h +++ b/lib/librte_port/rte_swx_port.h @@ -111,6 +111,90 @@ struct rte_swx_port_in_ops { rte_swx_port_in_stats_read_t stats_read; }; +/* + * Output port + */ + +/** + * Output port create + * + * @param[in] args + * Arguments for output port creation. Format specific to each port type. + * @return + * Handle to output port instance on success, NULL on error. + */ +typedef void * +(*rte_swx_port_out_create_t)(void *args); + +/** + * Output port free + * + * @param[in] args + * Output port handle. + */ +typedef void +(*rte_swx_port_out_free_t)(void *port); + +/** + * Output port packet transmit + * + * @param[in] port + * Output port handle. + * @param[in] pkt + * Packet to be transmitted. + */ +typedef void +(*rte_swx_port_out_pkt_tx_t)(void *port, + struct rte_swx_pkt *pkt); + +/** + * Output port flush + * + * @param[in] port + * Output port handle. + */ +typedef void +(*rte_swx_port_out_flush_t)(void *port); + +/** Output port statistics counters. */ +struct rte_swx_port_out_stats { + /** Number of packets. */ + uint64_t n_pkts; + + /** Number of bytes. */ + uint64_t n_bytes; +}; + +/** + * Output port statistics counters read + * + * @param[in] port + * Output port handle. + * @param[out] stats + * Output port statistics counters. Must point to valid memory. + */ +typedef void +(*rte_swx_port_out_stats_read_t)(void *port, + struct rte_swx_port_out_stats *stats); + +/** Output port operations. */ +struct rte_swx_port_out_ops { + /** Create. Must be non-NULL. */ + rte_swx_port_out_create_t create; + + /** Free. Must be non-NULL. */ + rte_swx_port_out_free_t free; + + /** Packet transmission. Must be non-NULL. */ + rte_swx_port_out_pkt_tx_t pkt_tx; + + /** Flush. May be NULL. */ + rte_swx_port_out_flush_t flush; + + /** Statistics counters read. Must be non-NULL. */ + rte_swx_port_out_stats_read_t stats_read; +}; + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 04/40] pipeline: add headers and meta-data 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (2 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 03/40] pipeline: add output port Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 05/40] pipeline: add extern objects and functions Cristian Dumitrescu ` (35 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add support for dynamically-defined packet headers and meta-data. The header and meta-data format are defined by the struct type they instantiate. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 3 + lib/librte_pipeline/rte_swx_pipeline.c | 413 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 85 ++++ 3 files changed, 501 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 88fd38ca8..6a48c3666 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,9 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_struct_type_register; + rte_swx_pipeline_packet_header_register; + rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 7aeac8cc8..cb2e32b83 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -20,6 +20,25 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +/* + * Struct. + */ +struct field { + char name[RTE_SWX_NAME_SIZE]; + uint32_t n_bits; + uint32_t offset; +}; + +struct struct_type { + TAILQ_ENTRY(struct_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct field *fields; + uint32_t n_fields; + uint32_t n_bits; +}; + +TAILQ_HEAD(struct_type_tailq, struct_type); + /* * Input port. */ @@ -71,24 +90,198 @@ struct port_out_runtime { void *obj; }; +/* + * Header. + */ +struct header { + TAILQ_ENTRY(header) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(header_tailq, header); + +struct header_runtime { + uint8_t *ptr0; +}; + +struct header_out_runtime { + uint8_t *ptr0; + uint8_t *ptr; + uint32_t n_bytes; +}; + /* * Pipeline. */ +struct thread { + /* Structures. */ + uint8_t **structs; + + /* Packet headers. */ + struct header_runtime *headers; /* Extracted or generated headers. */ + struct header_out_runtime *headers_out; /* Emitted headers. */ + uint8_t *header_storage; + uint8_t *header_out_storage; + uint64_t valid_headers; + uint32_t n_headers_out; + + /* Packet meta-data. */ + uint8_t *metadata; +}; + +#ifndef RTE_SWX_PIPELINE_THREADS_MAX +#define RTE_SWX_PIPELINE_THREADS_MAX 16 +#endif + struct rte_swx_pipeline { + struct struct_type_tailq struct_types; struct port_in_type_tailq port_in_types; struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct header_tailq headers; + struct struct_type *metadata_st; + uint32_t metadata_struct_id; struct port_in_runtime *in; struct port_out_runtime *out; + struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; + uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_headers; int build_done; int numa_node; }; +/* + * Struct. + */ +static struct struct_type * +struct_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct struct_type *elem; + + TAILQ_FOREACH(elem, &p->struct_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields) +{ + struct struct_type *st; + uint32_t i; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK(fields, EINVAL); + CHECK(n_fields, EINVAL); + + for (i = 0; i < n_fields; i++) { + struct rte_swx_field_params *f = &fields[i]; + uint32_t j; + + CHECK_NAME(f->name, EINVAL); + CHECK(f->n_bits, EINVAL); + CHECK(f->n_bits <= 64, EINVAL); + CHECK((f->n_bits & 7) == 0, EINVAL); + + for (j = 0; j < i; j++) { + struct rte_swx_field_params *f_prev = &fields[j]; + + CHECK(strcmp(f->name, f_prev->name), EINVAL); + } + } + + CHECK(!struct_type_find(p, name), EEXIST); + + /* Node allocation. */ + st = calloc(1, sizeof(struct struct_type)); + CHECK(st, ENOMEM); + + st->fields = calloc(n_fields, sizeof(struct field)); + if (!st->fields) { + free(st); + CHECK(0, ENOMEM); + } + + /* Node initialization. */ + strcpy(st->name, name); + for (i = 0; i < n_fields; i++) { + struct field *dst = &st->fields[i]; + struct rte_swx_field_params *src = &fields[i]; + + strcpy(dst->name, src->name); + dst->n_bits = src->n_bits; + dst->offset = st->n_bits; + + st->n_bits += src->n_bits; + } + st->n_fields = n_fields; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->struct_types, st, node); + + return 0; +} + +static int +struct_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + t->structs = calloc(p->n_structs, sizeof(uint8_t *)); + CHECK(t->structs, ENOMEM); + } + + return 0; +} + +static void +struct_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->structs); + t->structs = NULL; + } +} + +static void +struct_free(struct rte_swx_pipeline *p) +{ + struct_build_free(p); + + /* Struct types. */ + for ( ; ; ) { + struct struct_type *elem; + + elem = TAILQ_FIRST(&p->struct_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->struct_types, elem, node); + free(elem->fields); + free(elem); + } +} + /* * Input port. */ @@ -413,6 +606,205 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Header. + */ +static struct header * +header_find(struct rte_swx_pipeline *p, const char *name) +{ + struct header *elem; + + TAILQ_FOREACH(elem, &p->headers, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name) +{ + struct struct_type *st; + struct header *h; + size_t n_headers_max; + + CHECK(p, EINVAL); + CHECK_NAME(name, EINVAL); + CHECK_NAME(struct_type_name, EINVAL); + + CHECK(!header_find(p, name), EEXIST); + + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + + n_headers_max = RTE_SIZEOF_FIELD(struct thread, valid_headers) * 8; + CHECK(p->n_headers < n_headers_max, ENOSPC); + + /* Node allocation. */ + h = calloc(1, sizeof(struct header)); + CHECK(h, ENOMEM); + + /* Node initialization. */ + strcpy(h->name, name); + h->st = st; + h->struct_id = p->n_structs; + h->id = p->n_headers; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->headers, h, node); + p->n_headers++; + p->n_structs++; + + return 0; +} + +static int +header_build(struct rte_swx_pipeline *p) +{ + struct header *h; + uint32_t n_bytes = 0, i; + + TAILQ_FOREACH(h, &p->headers, node) { + n_bytes += h->st->n_bits / 8; + } + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t offset = 0; + + t->headers = calloc(p->n_headers, + sizeof(struct header_runtime)); + CHECK(t->headers, ENOMEM); + + t->headers_out = calloc(p->n_headers, + sizeof(struct header_out_runtime)); + CHECK(t->headers_out, ENOMEM); + + t->header_storage = calloc(1, n_bytes); + CHECK(t->header_storage, ENOMEM); + + t->header_out_storage = calloc(1, n_bytes); + CHECK(t->header_out_storage, ENOMEM); + + TAILQ_FOREACH(h, &p->headers, node) { + uint8_t *header_storage; + + header_storage = &t->header_storage[offset]; + offset += h->st->n_bits / 8; + + t->headers[h->id].ptr0 = header_storage; + t->structs[h->struct_id] = header_storage; + } + } + + return 0; +} + +static void +header_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->headers_out); + t->headers_out = NULL; + + free(t->headers); + t->headers = NULL; + + free(t->header_out_storage); + t->header_out_storage = NULL; + + free(t->header_storage); + t->header_storage = NULL; + } +} + +static void +header_free(struct rte_swx_pipeline *p) +{ + header_build_free(p); + + for ( ; ; ) { + struct header *elem; + + elem = TAILQ_FIRST(&p->headers); + if (!elem) + break; + + TAILQ_REMOVE(&p->headers, elem, node); + free(elem); + } +} + +/* + * Meta-data. + */ +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name) +{ + struct struct_type *st = NULL; + + CHECK(p, EINVAL); + + CHECK_NAME(struct_type_name, EINVAL); + st = struct_type_find(p, struct_type_name); + CHECK(st, EINVAL); + CHECK(!p->metadata_st, EINVAL); + + p->metadata_st = st; + p->metadata_struct_id = p->n_structs; + + p->n_structs++; + + return 0; +} + +static int +metadata_build(struct rte_swx_pipeline *p) +{ + uint32_t n_bytes = p->metadata_st->n_bits / 8; + uint32_t i; + + /* Thread-level initialization. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint8_t *metadata; + + metadata = calloc(1, n_bytes); + CHECK(metadata, ENOMEM); + + t->metadata = metadata; + t->structs[p->metadata_struct_id] = metadata; + } + + return 0; +} + +static void +metadata_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + free(t->metadata); + t->metadata = NULL; + } +} + +static void +metadata_free(struct rte_swx_pipeline *p) +{ + metadata_build_free(p); +} + /* * Pipeline. */ @@ -429,11 +821,14 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) CHECK(pipeline, ENOMEM); /* Initialization. */ + TAILQ_INIT(&pipeline->struct_types); TAILQ_INIT(&pipeline->port_in_types); TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->headers); + pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; *p = pipeline; @@ -446,8 +841,11 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + metadata_free(p); + header_free(p); port_out_free(p); port_in_free(p); + struct_free(p); free(p); } @@ -468,12 +866,27 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = struct_build(p); + if (status) + goto error; + + status = header_build(p); + if (status) + goto error; + + status = metadata_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + metadata_build_free(p); + header_build_free(p); port_out_build_free(p); port_in_build_free(p); + struct_build_free(p); return status; } diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2be83bd35..4a7b679a4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -147,6 +147,91 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Packet headers and meta-data + */ + +/** Structure (struct) field. */ +struct rte_swx_field_params { + /** Struct field name. */ + const char *name; + + /** Struct field size (in bits). + * Restriction: All struct fields must be a multiple of 8 bits. + * Restriction: All struct fields must be no greater than 64 bits. + */ + uint32_t n_bits; +}; + +/** + * Pipeline struct type register + * + * Structs are used extensively in many part of the pipeline to define the size + * and layout of a specific memory piece such as: headers, meta-data, action + * data stored in a table entry, mailboxes for extern objects and functions. + * Similar to C language structs, they are a well defined sequence of fields, + * with each field having a unique name and a constant size. + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Struct type name. + * @param[in] fields + * The sequence of struct fields. + * @param[in] n_fields + * The number of struct fields. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Struct type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_field_params *fields, + uint32_t n_fields); + +/** + * Pipeline packet header register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Header name. + * @param[in] struct_type_name + * The struct type instantiated by this packet header. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Header with this name already exists; + * -ENOSPC: Maximum number of headers reached for the pipeline. + */ +__rte_experimental +int +rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, + const char *name, + const char *struct_type_name); + +/** + * Pipeline packet meta-data register + * + * @param[in] p + * Pipeline handle. + * @param[in] struct_type_name + * The struct type instantiated by the packet meta-data. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, + const char *struct_type_name); + + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 05/40] pipeline: add extern objects and functions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (3 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 04/40] pipeline: add headers and meta-data Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 06/40] pipeline: add action Cristian Dumitrescu ` (34 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add extern objects and functions to plug in functions that cannot be efficiently implemented with existing instructions, e.g. special checksum/ECC, crypto, meters, stats arrays, heuristics, etc. In/out arguments are passed through mailbox with format defined by struct. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_extern.h | 98 ++++ lib/librte_pipeline/rte_swx_pipeline.c | 477 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 113 +++++ 6 files changed, 695 insertions(+), 1 deletion(-) create mode 100644 lib/librte_pipeline/rte_swx_extern.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 32582db9e..23bfd88e6 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index 880c2b274..bea406848 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -8,5 +8,6 @@ sources = files('rte_pipeline.c', headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', - 'rte_swx_pipeline.h',) + 'rte_swx_pipeline.h', + 'rte_swx_extern.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 6a48c3666..4297e185d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -60,6 +60,10 @@ EXPERIMENTAL { rte_swx_pipeline_port_in_config; rte_swx_pipeline_port_out_type_register; rte_swx_pipeline_port_out_config; + rte_swx_pipeline_extern_type_register; + rte_swx_pipeline_extern_type_member_func_register; + rte_swx_pipeline_extern_object_config; + rte_swx_pipeline_extern_func_register; rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; diff --git a/lib/librte_pipeline/rte_swx_extern.h b/lib/librte_pipeline/rte_swx_extern.h new file mode 100644 index 000000000..e10e963d6 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_extern.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_EXTERN_H__ +#define __INCLUDE_RTE_SWX_EXTERN_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Extern objects and functions + * + * Extern object and extern function interfaces. The extern objects and extern + * functions provide the mechanisms to hook external functionality into the + * packet processing pipeline. + */ + +#include <stdint.h> + +/* + * Extern type + */ + +/** + * Extern object constructor + * + * @param[in] args + * Extern object constructor arguments. It may be NULL. + * @return + * Extern object handle. + */ +typedef void * +(*rte_swx_extern_type_constructor_t)(const char *args); + +/** + * Extern object destructor + * + * @param[in] object + * Extern object handle. + */ +typedef void +(*rte_swx_extern_type_destructor_t)(void *object); + +/** + * Extern object member function + * + * The mailbox is used to pass input arguments to the member function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same member function for the same extern object. + * + * Multiple invocations of the same member function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the member + * function must be invoked again with exactly the same object and mailbox + * arguments. + * + * @param[in] object + * Extern object handle. + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_type_member_func_t)(void *object, void *mailbox); + +/* + * Extern function + */ + +/** The mailbox is used to pass input arguments to the extern function and + * retrieve the output results. The mailbox mechanism allows for multiple + * concurrent executions of the same extern function. + * + * Multiple invocations of the same extern function may be required in order for + * the associated operation to complete. The completion is flagged by a return + * value of 1, in which case the results are available in the mailbox; in case + * of a return value of 0, the operation is not yet completed, so the extern + * function must be invoked again with exactly the same mailbox argument. + * + * @param[in] mailbox + * Extern object mailbox. + * @return + * 0 when the operation is not yet completed, and 1 when the operation is + * completed. No other return values are allowed. + */ +typedef int +(*rte_swx_extern_func_t)(void *mailbox); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index cb2e32b83..2335831bf 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -90,6 +90,70 @@ struct port_out_runtime { void *obj; }; +/* + * Extern object. + */ +struct extern_type_member_func { + TAILQ_ENTRY(extern_type_member_func) node; + char name[RTE_SWX_NAME_SIZE]; + rte_swx_extern_type_member_func_t func; + uint32_t id; +}; + +TAILQ_HEAD(extern_type_member_func_tailq, extern_type_member_func); + +struct extern_type { + TAILQ_ENTRY(extern_type) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_type_constructor_t constructor; + rte_swx_extern_type_destructor_t destructor; + struct extern_type_member_func_tailq funcs; + uint32_t n_funcs; +}; + +TAILQ_HEAD(extern_type_tailq, extern_type); + +struct extern_obj { + TAILQ_ENTRY(extern_obj) node; + char name[RTE_SWX_NAME_SIZE]; + struct extern_type *type; + void *obj; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_obj_tailq, extern_obj); + +#ifndef RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX +#define RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX 8 +#endif + +struct extern_obj_runtime { + void *obj; + uint8_t *mailbox; + rte_swx_extern_type_member_func_t funcs[RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX]; +}; + +/* + * Extern function. + */ +struct extern_func { + TAILQ_ENTRY(extern_func) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *mailbox_struct_type; + rte_swx_extern_func_t func; + uint32_t struct_id; + uint32_t id; +}; + +TAILQ_HEAD(extern_func_tailq, extern_func); + +struct extern_func_runtime { + uint8_t *mailbox; + rte_swx_extern_func_t func; +}; + /* * Header. */ @@ -130,6 +194,10 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + + /* Extern objects and functions. */ + struct extern_obj_runtime *extern_objs; + struct extern_func_runtime *extern_funcs; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -142,6 +210,9 @@ struct rte_swx_pipeline { struct port_in_tailq ports_in; struct port_out_type_tailq port_out_types; struct port_out_tailq ports_out; + struct extern_type_tailq extern_types; + struct extern_obj_tailq extern_objs; + struct extern_func_tailq extern_funcs; struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; @@ -153,6 +224,8 @@ struct rte_swx_pipeline { uint32_t n_structs; uint32_t n_ports_in; uint32_t n_ports_out; + uint32_t n_extern_objs; + uint32_t n_extern_funcs; uint32_t n_headers; int build_done; int numa_node; @@ -606,6 +679,395 @@ port_out_free(struct rte_swx_pipeline *p) } } +/* + * Extern object. + */ +static struct extern_type * +extern_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_type *elem; + + TAILQ_FOREACH(elem, &p->extern_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_type_member_func * +extern_type_member_func_find(struct extern_type *type, const char *name) +{ + struct extern_type_member_func *elem; + + TAILQ_FOREACH(elem, &type->funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct extern_obj * +extern_obj_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_obj *elem; + + TAILQ_FOREACH(elem, &p->extern_objs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor) +{ + struct extern_type *elem; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_type_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(constructor, EINVAL); + CHECK(destructor, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct extern_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->mailbox_struct_type = mailbox_struct_type; + elem->constructor = constructor; + elem->destructor = destructor; + TAILQ_INIT(&elem->funcs); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_types, elem, node); + + return 0; +} + +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func) +{ + struct extern_type *type; + struct extern_type_member_func *type_member; + + CHECK(p, EINVAL); + + CHECK(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + CHECK(type->n_funcs < RTE_SWX_EXTERN_TYPE_MEMBER_FUNCS_MAX, ENOSPC); + + CHECK(name, EINVAL); + CHECK(!extern_type_member_func_find(type, name), EEXIST); + + CHECK(member_func, EINVAL); + + /* Node allocation. */ + type_member = calloc(1, sizeof(struct extern_type_member_func)); + CHECK(type_member, ENOMEM); + + /* Node initialization. */ + strcpy(type_member->name, name); + type_member->func = member_func; + type_member->id = type->n_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&type->funcs, type_member, node); + type->n_funcs++; + + return 0; +} + +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args) +{ + struct extern_type *type; + struct extern_obj *obj; + void *obj_handle; + + CHECK(p, EINVAL); + + CHECK_NAME(extern_type_name, EINVAL); + type = extern_type_find(p, extern_type_name); + CHECK(type, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_obj_find(p, name), EEXIST); + + /* Node allocation. */ + obj = calloc(1, sizeof(struct extern_obj)); + CHECK(obj, ENOMEM); + + /* Object construction. */ + obj_handle = type->constructor(args); + if (!obj_handle) { + free(obj); + CHECK(0, ENODEV); + } + + /* Node initialization. */ + strcpy(obj->name, name); + obj->type = type; + obj->obj = obj_handle; + obj->struct_id = p->n_structs; + obj->id = p->n_extern_objs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_objs, obj, node); + p->n_extern_objs++; + p->n_structs++; + + return 0; +} + +static int +extern_obj_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_obj *obj; + + t->extern_objs = calloc(p->n_extern_objs, + sizeof(struct extern_obj_runtime)); + CHECK(t->extern_objs, ENOMEM); + + TAILQ_FOREACH(obj, &p->extern_objs, node) { + struct extern_obj_runtime *r = + &t->extern_objs[obj->id]; + struct extern_type_member_func *func; + uint32_t mailbox_size = + obj->type->mailbox_struct_type->n_bits / 8; + + r->obj = obj->obj; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + TAILQ_FOREACH(func, &obj->type->funcs, node) + r->funcs[func->id] = func->func; + + t->structs[obj->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_obj_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_objs) + continue; + + for (j = 0; j < p->n_extern_objs; j++) { + struct extern_obj_runtime *r = &t->extern_objs[j]; + + free(r->mailbox); + } + + free(t->extern_objs); + t->extern_objs = NULL; + } +} + +static void +extern_obj_free(struct rte_swx_pipeline *p) +{ + extern_obj_build_free(p); + + /* Extern objects. */ + for ( ; ; ) { + struct extern_obj *elem; + + elem = TAILQ_FIRST(&p->extern_objs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_objs, elem, node); + if (elem->obj) + elem->type->destructor(elem->obj); + free(elem); + } + + /* Extern types. */ + for ( ; ; ) { + struct extern_type *elem; + + elem = TAILQ_FIRST(&p->extern_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_types, elem, node); + + for ( ; ; ) { + struct extern_type_member_func *func; + + func = TAILQ_FIRST(&elem->funcs); + if (!func) + break; + + TAILQ_REMOVE(&elem->funcs, func, node); + free(func); + } + + free(elem); + } +} + +/* + * Extern function. + */ +static struct extern_func * +extern_func_find(struct rte_swx_pipeline *p, const char *name) +{ + struct extern_func *elem; + + TAILQ_FOREACH(elem, &p->extern_funcs, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func) +{ + struct extern_func *f; + struct struct_type *mailbox_struct_type; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!extern_func_find(p, name), EEXIST); + + CHECK_NAME(mailbox_struct_type_name, EINVAL); + mailbox_struct_type = struct_type_find(p, mailbox_struct_type_name); + CHECK(mailbox_struct_type, EINVAL); + + CHECK(func, EINVAL); + + /* Node allocation. */ + f = calloc(1, sizeof(struct extern_func)); + CHECK(func, ENOMEM); + + /* Node initialization. */ + strcpy(f->name, name); + f->mailbox_struct_type = mailbox_struct_type; + f->func = func; + f->struct_id = p->n_structs; + f->id = p->n_extern_funcs; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->extern_funcs, f, node); + p->n_extern_funcs++; + p->n_structs++; + + return 0; +} + +static int +extern_func_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct extern_func *func; + + /* Memory allocation. */ + t->extern_funcs = calloc(p->n_extern_funcs, + sizeof(struct extern_func_runtime)); + CHECK(t->extern_funcs, ENOMEM); + + /* Extern function. */ + TAILQ_FOREACH(func, &p->extern_funcs, node) { + struct extern_func_runtime *r = + &t->extern_funcs[func->id]; + uint32_t mailbox_size = + func->mailbox_struct_type->n_bits / 8; + + r->func = func->func; + + r->mailbox = calloc(1, mailbox_size); + CHECK(r->mailbox, ENOMEM); + + t->structs[func->struct_id] = r->mailbox; + } + } + + return 0; +} + +static void +extern_func_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->extern_funcs) + continue; + + for (j = 0; j < p->n_extern_funcs; j++) { + struct extern_func_runtime *r = &t->extern_funcs[j]; + + free(r->mailbox); + } + + free(t->extern_funcs); + t->extern_funcs = NULL; + } +} + +static void +extern_func_free(struct rte_swx_pipeline *p) +{ + extern_func_build_free(p); + + for ( ; ; ) { + struct extern_func *elem; + + elem = TAILQ_FIRST(&p->extern_funcs); + if (!elem) + break; + + TAILQ_REMOVE(&p->extern_funcs, elem, node); + free(elem); + } +} + /* * Header. */ @@ -826,6 +1288,9 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->ports_in); TAILQ_INIT(&pipeline->port_out_types); TAILQ_INIT(&pipeline->ports_out); + TAILQ_INIT(&pipeline->extern_types); + TAILQ_INIT(&pipeline->extern_objs); + TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ @@ -843,6 +1308,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) metadata_free(p); header_free(p); + extern_func_free(p); + extern_obj_free(p); port_out_free(p); port_in_free(p); struct_free(p); @@ -870,6 +1337,14 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = extern_obj_build(p); + if (status) + goto error; + + status = extern_func_build(p); + if (status) + goto error; + status = header_build(p); if (status) goto error; @@ -884,6 +1359,8 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) error: metadata_build_free(p); header_build_free(p); + extern_func_build_free(p); + extern_obj_build_free(p); port_out_build_free(p); port_in_build_free(p); struct_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 4a7b679a4..2e8a6cdf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_extern.h" /** Name size. */ #ifndef RTE_SWX_NAME_SIZE @@ -147,6 +148,118 @@ rte_swx_pipeline_port_out_config(struct rte_swx_pipeline *p, const char *port_type_name, void *args); +/* + * Extern objects and functions + */ + +/** + * Pipeline extern type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern type name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * the extern objects that are instances of this type. Each extern object gets + * its own mailbox, which is used to pass the input arguments to the member + * functions and retrieve the output results. + * @param[in] constructor + * Function used to create the extern objects that are instances of this type. + * @param[in] destructor + * Function used to free the extern objects that are instances of this type. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_type_constructor_t constructor, + rte_swx_extern_type_destructor_t destructor); + +/** + * Pipeline extern type member function register + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new member function to be added to the extern type. + * @param[in] member_func + * The new member function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Member function with this name already exists for this type; + * -ENOSPC: Maximum number of member functions reached for this type. + */ +__rte_experimental +int +rte_swx_pipeline_extern_type_member_func_register(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + rte_swx_extern_type_member_func_t member_func); + +/** + * Pipeline extern object configure + * + * Instantiate a given extern type to create new extern object. + * + * @param[in] p + * Pipeline handle. + * @param[in] extern_type_name + * Existing extern type name. + * @param[in] name + * Name for the new object instantiating the extern type. + * @param[in] args + * Extern object constructor arguments. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern object with this name already exists; + * -ENODEV: Extern object constructor error. + */ +__rte_experimental +int +rte_swx_pipeline_extern_object_config(struct rte_swx_pipeline *p, + const char *extern_type_name, + const char *name, + const char *args); + +/** + * Pipeline extern function register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Extern function name. + * @param[in] mailbox_struct_type_name + * Name of existing struct type used to define the mailbox size and layout for + * this extern function. The mailbox is used to pass the input arguments to + * the extern function and retrieve the output results. + * @param[in] func + * The extern function. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Extern function with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, + const char *name, + const char *mailbox_struct_type_name, + rte_swx_extern_func_t func); + /* * Packet headers and meta-data */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 06/40] pipeline: add action 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (4 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 05/40] pipeline: add extern objects and functions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 07/40] pipeline: add tables Cristian Dumitrescu ` (33 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add actions that are dynamically-defined through instructions as opposed to pre-defined. The actions are subroutines of the pipeline program that triggered by table lookup. The input arguments are the action data from the table entry (format defined by struct), the headers and meta-data are in/out. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 147 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 32 ++++ 3 files changed, 180 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 4297e185d..c701f158d 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -67,6 +67,7 @@ EXPERIMENTAL { rte_swx_pipeline_struct_type_register; rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; + rte_swx_pipeline_action_config; rte_swx_pipeline_build; rte_swx_pipeline_free; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 2335831bf..678700050 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -177,6 +177,26 @@ struct header_out_runtime { uint32_t n_bytes; }; +/* + * Instruction. + */ +struct instruction { +}; + +/* + * Action. + */ +struct action { + TAILQ_ENTRY(action) node; + char name[RTE_SWX_NAME_SIZE]; + struct struct_type *st; + struct instruction *instructions; + uint32_t n_instructions; + uint32_t id; +}; + +TAILQ_HEAD(action_tailq, action); + /* * Pipeline. */ @@ -216,9 +236,11 @@ struct rte_swx_pipeline { struct header_tailq headers; struct struct_type *metadata_st; uint32_t metadata_struct_id; + struct action_tailq actions; struct port_in_runtime *in; struct port_out_runtime *out; + struct instruction **action_instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -226,6 +248,7 @@ struct rte_swx_pipeline { uint32_t n_ports_out; uint32_t n_extern_objs; uint32_t n_extern_funcs; + uint32_t n_actions; uint32_t n_headers; int build_done; int numa_node; @@ -1267,6 +1290,123 @@ metadata_free(struct rte_swx_pipeline *p) metadata_build_free(p); } +/* + * Instruction. + */ +static int +instruction_config(struct rte_swx_pipeline *p __rte_unused, + struct action *a __rte_unused, + const char **instructions __rte_unused, + uint32_t n_instructions __rte_unused) +{ + return 0; +} + +/* + * Action. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name) +{ + struct action *elem; + + if (!name) + return NULL; + + TAILQ_FOREACH(elem, &p->actions, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions) +{ + struct struct_type *args_struct_type; + struct action *a; + int err; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!action_find(p, name), EEXIST); + + if (args_struct_type_name) { + CHECK_NAME(args_struct_type_name, EINVAL); + args_struct_type = struct_type_find(p, args_struct_type_name); + CHECK(args_struct_type, EINVAL); + } else { + args_struct_type = NULL; + } + + /* Node allocation. */ + a = calloc(1, sizeof(struct action)); + CHECK(a, ENOMEM); + + /* Node initialization. */ + strcpy(a->name, name); + a->st = args_struct_type; + a->id = p->n_actions; + + /* Instruction translation. */ + err = instruction_config(p, a, instructions, n_instructions); + if (err) { + free(a); + return err; + } + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->actions, a, node); + p->n_actions++; + + return 0; +} + +static int +action_build(struct rte_swx_pipeline *p) +{ + struct action *action; + + p->action_instructions = calloc(p->n_actions, + sizeof(struct instruction *)); + CHECK(p->action_instructions, ENOMEM); + + TAILQ_FOREACH(action, &p->actions, node) + p->action_instructions[action->id] = action->instructions; + + return 0; +} + +static void +action_build_free(struct rte_swx_pipeline *p) +{ + free(p->action_instructions); + p->action_instructions = NULL; +} + +static void +action_free(struct rte_swx_pipeline *p) +{ + action_build_free(p); + + for ( ; ; ) { + struct action *action; + + action = TAILQ_FIRST(&p->actions); + if (!action) + break; + + TAILQ_REMOVE(&p->actions, action, node); + free(action->instructions); + free(action); + } +} + /* * Pipeline. */ @@ -1292,6 +1432,7 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_objs); TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); + TAILQ_INIT(&pipeline->actions); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1306,6 +1447,7 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + action_free(p); metadata_free(p); header_free(p); extern_func_free(p); @@ -1353,10 +1495,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = action_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + action_build_free(p); metadata_build_free(p); header_build_free(p); extern_func_build_free(p); diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 2e8a6cdf8..1b20293cb 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -344,6 +344,38 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Pipeline action + */ + +/** + * Pipeline action configure + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Action name. + * @param[in] args_struct_type_name + * The struct type instantiated by the action data. The action data represent + * the action arguments that are stored in the table entry together with the + * action ID. Set to NULL when the action does not have any arguments. + * @param[in] instructions + * Action instructions. + * @param[in] n_instructions + * Number of action instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Action with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, + const char *name, + const char *args_struct_type_name, + const char **instructions, + uint32_t n_instructions); /** * Pipeline build -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 07/40] pipeline: add tables 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (5 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 06/40] pipeline: add action Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 08/40] pipeline: add pipeline instructions Cristian Dumitrescu ` (32 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add tables to the pipeline. The match fields are flexibly selected from the headers and meta-data. The set of actions is flexibly selected per table from the pipeline set. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 4 + lib/librte_pipeline/rte_swx_ctl.h | 85 +++ lib/librte_pipeline/rte_swx_pipeline.c | 700 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 118 ++++ lib/librte_table/Makefile | 1 + lib/librte_table/meson.build | 3 +- lib/librte_table/rte_swx_table.h | 295 ++++++++ 9 files changed, 1208 insertions(+), 2 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.h create mode 100644 lib/librte_table/rte_swx_table.h diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index 23bfd88e6..d214b1aeb 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -27,5 +27,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_extern.h SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_pipeline.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_swx_ctl.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index bea406848..d5f4d16e5 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -9,5 +9,6 @@ headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', 'rte_swx_pipeline.h', - 'rte_swx_extern.h',) + 'rte_swx_extern.h', + 'rte_swx_ctl.h',) deps += ['port', 'table', 'meter', 'sched', 'cryptodev'] diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index c701f158d..b9e59bce2 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -68,6 +68,10 @@ EXPERIMENTAL { rte_swx_pipeline_packet_header_register; rte_swx_pipeline_packet_metadata_register; rte_swx_pipeline_action_config; + rte_swx_pipeline_table_type_register; + rte_swx_pipeline_table_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h new file mode 100644 index 000000000..c824ab56f --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_CTL_H__ +#define __INCLUDE_RTE_SWX_CTL_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Pipeline Control + */ + +#include <stddef.h> +#include <stdint.h> + +#include <rte_compat.h> + +#include "rte_swx_table.h" + +/* + * Table Update API. + */ + +/** Table state. */ +struct rte_swx_table_state { + /** Table object. */ + void *obj; + + /** Action ID of the table default action. */ + uint64_t default_action_id; + + /** Action data of the table default action. Ignored when the action + * data size is zero; otherwise, action data size bytes are meaningful. + */ + uint8_t *default_action_data; +}; + +/** + * Pipeline table state get + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the *table_state* contains the pointer to the + * current pipeline table state, which is an array of *n_tables* elements, + * with array element i containing the state of the i-th pipeline table. The + * pipeline continues to own all the data structures directly or indirectly + * referenced by the *table_state* until the subsequent successful invocation + * of function *rte_swx_pipeline_table_state_set*. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state); + +/** + * Pipeline table state set + * + * @param[in] p + * Pipeline handle. + * @param[out] table_state + * After successful execution, the pipeline table state is updated to this + * *table_state*. The ownership of all the data structures directly or + * indirectly referenced by this *table_state* is passed from the caller to + * the pipeline. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 678700050..43cdb0f7c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -10,6 +10,7 @@ #include <rte_common.h> #include "rte_swx_pipeline.h" +#include "rte_swx_ctl.h" #define CHECK(condition, err_code) \ do { \ @@ -197,6 +198,55 @@ struct action { TAILQ_HEAD(action_tailq, action); +/* + * Table. + */ +struct table_type { + TAILQ_ENTRY(table_type) node; + char name[RTE_SWX_NAME_SIZE]; + enum rte_swx_table_match_type match_type; + struct rte_swx_table_ops ops; +}; + +TAILQ_HEAD(table_type_tailq, table_type); + +struct match_field { + enum rte_swx_table_match_type match_type; + struct field *field; +}; + +struct table { + TAILQ_ENTRY(table) node; + char name[RTE_SWX_NAME_SIZE]; + char args[RTE_SWX_NAME_SIZE]; + struct table_type *type; /* NULL when n_fields == 0. */ + + /* Match. */ + struct match_field *fields; + uint32_t n_fields; + int is_header; /* Only valid when n_fields > 0. */ + struct header *header; /* Only valid when n_fields > 0. */ + + /* Action. */ + struct action **actions; + struct action *default_action; + uint8_t *default_action_data; + uint32_t n_actions; + int default_action_is_const; + uint32_t action_data_size_max; + + uint32_t size; + uint32_t id; +}; + +TAILQ_HEAD(table_tailq, table); + +struct table_runtime { + rte_swx_table_lookup_t func; + void *mailbox; + uint8_t **key; +}; + /* * Pipeline. */ @@ -215,6 +265,12 @@ struct thread { /* Packet meta-data. */ uint8_t *metadata; + /* Tables. */ + struct table_runtime *tables; + struct rte_swx_table_state *table_state; + uint64_t action_id; + int hit; /* 0 = Miss, 1 = Hit. */ + /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; @@ -237,10 +293,13 @@ struct rte_swx_pipeline { struct struct_type *metadata_st; uint32_t metadata_struct_id; struct action_tailq actions; + struct table_type_tailq table_types; + struct table_tailq tables; struct port_in_runtime *in; struct port_out_runtime *out; struct instruction **action_instructions; + struct rte_swx_table_state *table_state; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -249,6 +308,7 @@ struct rte_swx_pipeline { uint32_t n_extern_objs; uint32_t n_extern_funcs; uint32_t n_actions; + uint32_t n_tables; uint32_t n_headers; int build_done; int numa_node; @@ -269,6 +329,21 @@ struct_type_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +struct_type_field_find(struct struct_type *st, const char *name) +{ + uint32_t i; + + for (i = 0; i < st->n_fields; i++) { + struct field *f = &st->fields[i]; + + if (strcmp(f->name, name) == 0) + return f; + } + + return NULL; +} + int rte_swx_pipeline_struct_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1106,6 +1181,50 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +header_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct header **header) +{ + struct header *h; + struct field *f; + char *header_name, *field_name; + + if ((name[0] != 'h') || (name[1] != '.')) + return NULL; + + header_name = strdup(&name[2]); + if (!header_name) + return NULL; + + field_name = strchr(header_name, '.'); + if (!field_name) { + free(header_name); + return NULL; + } + + *field_name = 0; + field_name++; + + h = header_find(p, header_name); + if (!h) { + free(header_name); + return NULL; + } + + f = struct_type_field_find(h->st, field_name); + if (!f) { + free(header_name); + return NULL; + } + + if (header) + *header = h; + + free(header_name); + return f; +} + int rte_swx_pipeline_packet_header_register(struct rte_swx_pipeline *p, const char *name, @@ -1229,6 +1348,18 @@ header_free(struct rte_swx_pipeline *p) /* * Meta-data. */ +static struct field * +metadata_field_parse(struct rte_swx_pipeline *p, const char *name) +{ + if (!p->metadata_st) + return NULL; + + if (name[0] != 'm' || name[1] != '.') + return NULL; + + return struct_type_field_find(p->metadata_st, &name[2]); +} + int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name) @@ -1407,6 +1538,536 @@ action_free(struct rte_swx_pipeline *p) } } +/* + * Table. + */ +static struct table_type * +table_type_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table_type *elem; + + TAILQ_FOREACH(elem, &p->table_types, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table_type * +table_type_resolve(struct rte_swx_pipeline *p, + const char *recommended_type_name, + enum rte_swx_table_match_type match_type) +{ + struct table_type *elem; + + /* Only consider the recommended type if the match type is correct. */ + if (recommended_type_name) + TAILQ_FOREACH(elem, &p->table_types, node) + if (!strcmp(elem->name, recommended_type_name) && + (elem->match_type == match_type)) + return elem; + + /* Ignore the recommended type and get the first element with this match + * type. + */ + TAILQ_FOREACH(elem, &p->table_types, node) + if (elem->match_type == match_type) + return elem; + + return NULL; +} + +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name) +{ + struct table *elem; + + TAILQ_FOREACH(elem, &p->tables, node) + if (strcmp(elem->name, name) == 0) + return elem; + + return NULL; +} + +static struct table * +table_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct table *table = NULL; + + TAILQ_FOREACH(table, &p->tables, node) + if (table->id == id) + return table; + + return NULL; +} + +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops) +{ + struct table_type *elem; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_type_find(p, name), EEXIST); + + CHECK(ops, EINVAL); + CHECK(ops->create, EINVAL); + CHECK(ops->lkp, EINVAL); + CHECK(ops->free, EINVAL); + + /* Node allocation. */ + elem = calloc(1, sizeof(struct table_type)); + CHECK(elem, ENOMEM); + + /* Node initialization. */ + strcpy(elem->name, name); + elem->match_type = match_type; + memcpy(&elem->ops, ops, sizeof(*ops)); + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->table_types, elem, node); + + return 0; +} + +static enum rte_swx_table_match_type +table_match_type_resolve(struct rte_swx_match_field_params *fields, + uint32_t n_fields) +{ + uint32_t i; + + for (i = 0; i < n_fields; i++) + if (fields[i].match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + + if (i == n_fields) + return RTE_SWX_TABLE_MATCH_EXACT; + + if ((i == n_fields - 1) && + (fields[i].match_type == RTE_SWX_TABLE_MATCH_LPM)) + return RTE_SWX_TABLE_MATCH_LPM; + + return RTE_SWX_TABLE_MATCH_WILDCARD; +} + +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size) +{ + struct table_type *type; + struct table *t; + struct action *default_action; + struct header *header = NULL; + int is_header = 0; + uint32_t offset_prev = 0, action_data_size_max = 0, i; + + CHECK(p, EINVAL); + + CHECK_NAME(name, EINVAL); + CHECK(!table_find(p, name), EEXIST); + + CHECK(params, EINVAL); + + /* Match checks. */ + CHECK(!params->n_fields || params->fields, EINVAL); + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct header *h; + struct field *hf, *mf; + uint32_t offset; + + CHECK_NAME(field->name, EINVAL); + + hf = header_field_parse(p, field->name, &h); + mf = metadata_field_parse(p, field->name); + offset = hf ? hf->offset : mf->offset; + + CHECK(hf || mf, EINVAL); + + if (i == 0) { + is_header = hf ? 1 : 0; + header = hf ? h : NULL; + offset_prev = offset; + + continue; + } + + CHECK((is_header && hf && (h->id == header->id)) || + (!is_header && mf), EINVAL); + + CHECK(offset > offset_prev, EINVAL); + offset_prev = offset; + } + + /* Action checks. */ + CHECK(params->n_actions, EINVAL); + CHECK(params->action_names, EINVAL); + for (i = 0; i < params->n_actions; i++) { + const char *action_name = params->action_names[i]; + struct action *a; + uint32_t action_data_size; + + CHECK(action_name, EINVAL); + + a = action_find(p, action_name); + CHECK(a, EINVAL); + + action_data_size = a->st ? a->st->n_bits / 8 : 0; + if (action_data_size > action_data_size_max) + action_data_size_max = action_data_size; + } + + CHECK(params->default_action_name, EINVAL); + for (i = 0; i < p->n_actions; i++) + if (!strcmp(params->action_names[i], + params->default_action_name)) + break; + CHECK(i < params->n_actions, EINVAL); + default_action = action_find(p, params->default_action_name); + CHECK((default_action->st && params->default_action_data) || + !params->default_action_data, EINVAL); + + /* Table type checks. */ + if (params->n_fields) { + enum rte_swx_table_match_type match_type; + + match_type = table_match_type_resolve(params->fields, + params->n_fields); + type = table_type_resolve(p, + recommended_table_type_name, + match_type); + CHECK(type, EINVAL); + } else { + type = NULL; + } + + /* Memory allocation. */ + t = calloc(1, sizeof(struct table)); + CHECK(t, ENOMEM); + + t->fields = calloc(params->n_fields, sizeof(struct match_field)); + if (!t->fields) { + free(t); + CHECK(0, ENOMEM); + } + + t->actions = calloc(params->n_actions, sizeof(struct action *)); + if (!t->actions) { + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + + if (action_data_size_max) { + t->default_action_data = calloc(1, action_data_size_max); + if (!t->default_action_data) { + free(t->actions); + free(t->fields); + free(t); + CHECK(0, ENOMEM); + } + } + + /* Node initialization. */ + strcpy(t->name, name); + if (args && args[0]) + strcpy(t->args, args); + t->type = type; + + for (i = 0; i < params->n_fields; i++) { + struct rte_swx_match_field_params *field = ¶ms->fields[i]; + struct match_field *f = &t->fields[i]; + + f->match_type = field->match_type; + f->field = is_header ? + header_field_parse(p, field->name, NULL) : + metadata_field_parse(p, field->name); + } + t->n_fields = params->n_fields; + t->is_header = is_header; + t->header = header; + + for (i = 0; i < params->n_actions; i++) + t->actions[i] = action_find(p, params->action_names[i]); + t->default_action = default_action; + if (default_action->st) + memcpy(t->default_action_data, + params->default_action_data, + default_action->st->n_bits / 8); + t->n_actions = params->n_actions; + t->default_action_is_const = params->default_action_is_const; + t->action_data_size_max = action_data_size_max; + + t->size = size; + t->id = p->n_tables; + + /* Node add to tailq. */ + TAILQ_INSERT_TAIL(&p->tables, t, node); + p->n_tables++; + + return 0; +} + +static struct rte_swx_table_params * +table_params_get(struct table *table) +{ + struct rte_swx_table_params *params; + struct field *first, *last; + uint8_t *key_mask; + uint32_t key_size, key_offset, action_data_size, i; + + /* Memory allocation. */ + params = calloc(1, sizeof(struct rte_swx_table_params)); + if (!params) + return NULL; + + /* Key offset and size. */ + first = table->fields[0].field; + last = table->fields[table->n_fields - 1].field; + key_offset = first->offset / 8; + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* Memory allocation. */ + key_mask = calloc(1, key_size); + if (!key_mask) { + free(params); + return NULL; + } + + /* Key mask. */ + for (i = 0; i < table->n_fields; i++) { + struct field *f = table->fields[i].field; + uint32_t start = (f->offset - first->offset) / 8; + size_t size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + + /* Action data size. */ + action_data_size = 0; + for (i = 0; i < table->n_actions; i++) { + struct action *action = table->actions[i]; + uint32_t ads = action->st ? action->st->n_bits / 8 : 0; + + if (ads > action_data_size) + action_data_size = ads; + } + + /* Fill in. */ + params->match_type = table->type->match_type; + params->key_size = key_size; + params->key_offset = key_offset; + params->key_mask0 = key_mask; + params->action_data_size = action_data_size; + params->n_keys_max = table->size; + + return params; +} + +static void +table_params_free(struct rte_swx_table_params *params) +{ + if (!params) + return; + + free(params->key_mask0); + free(params); +} + +static int +table_state_build(struct rte_swx_pipeline *p) +{ + struct table *table; + + p->table_state = calloc(p->n_tables, + sizeof(struct rte_swx_table_state)); + CHECK(p->table_state, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct rte_swx_table_state *ts = &p->table_state[table->id]; + + if (table->type) { + struct rte_swx_table_params *params; + + /* ts->obj. */ + params = table_params_get(table); + CHECK(params, ENOMEM); + + ts->obj = table->type->ops.create(params, + NULL, + table->args, + p->numa_node); + + table_params_free(params); + CHECK(ts->obj, ENODEV); + } + + /* ts->default_action_data. */ + if (table->action_data_size_max) { + ts->default_action_data = + malloc(table->action_data_size_max); + CHECK(ts->default_action_data, ENOMEM); + + memcpy(ts->default_action_data, + table->default_action_data, + table->action_data_size_max); + } + + /* ts->default_action_id. */ + ts->default_action_id = table->default_action->id; + } + + return 0; +} + +static void +table_state_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + if (!p->table_state) + return; + + for (i = 0; i < p->n_tables; i++) { + struct rte_swx_table_state *ts = &p->table_state[i]; + struct table *table = table_find_by_id(p, i); + + /* ts->obj. */ + if (table->type && ts->obj) + table->type->ops.free(ts->obj); + + /* ts->default_action_data. */ + free(ts->default_action_data); + } + + free(p->table_state); + p->table_state = NULL; +} + +static void +table_state_free(struct rte_swx_pipeline *p) +{ + table_state_build_free(p); +} + +static int +table_stub_lkp(void *table __rte_unused, + void *mailbox __rte_unused, + uint8_t **key __rte_unused, + uint64_t *action_id __rte_unused, + uint8_t **action_data __rte_unused, + int *hit) +{ + *hit = 0; + return 1; /* DONE. */ +} + +static int +table_build(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + struct table *table; + + t->tables = calloc(p->n_tables, sizeof(struct table_runtime)); + CHECK(t->tables, ENOMEM); + + TAILQ_FOREACH(table, &p->tables, node) { + struct table_runtime *r = &t->tables[table->id]; + + if (table->type) { + uint64_t size; + + size = table->type->ops.mailbox_size_get(); + + /* r->func. */ + r->func = table->type->ops.lkp; + + /* r->mailbox. */ + if (size) { + r->mailbox = calloc(1, size); + CHECK(r->mailbox, ENOMEM); + } + + /* r->key. */ + r->key = table->is_header ? + &t->structs[table->header->struct_id] : + &t->structs[p->metadata_struct_id]; + } else { + r->func = table_stub_lkp; + } + } + } + + return 0; +} + +static void +table_build_free(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + uint32_t j; + + if (!t->tables) + continue; + + for (j = 0; j < p->n_tables; j++) { + struct table_runtime *r = &t->tables[j]; + + free(r->mailbox); + } + + free(t->tables); + t->tables = NULL; + } +} + +static void +table_free(struct rte_swx_pipeline *p) +{ + table_build_free(p); + + /* Tables. */ + for ( ; ; ) { + struct table *elem; + + elem = TAILQ_FIRST(&p->tables); + if (!elem) + break; + + TAILQ_REMOVE(&p->tables, elem, node); + free(elem->fields); + free(elem->actions); + free(elem->default_action_data); + free(elem); + } + + /* Table types. */ + for ( ; ; ) { + struct table_type *elem; + + elem = TAILQ_FIRST(&p->table_types); + if (!elem) + break; + + TAILQ_REMOVE(&p->table_types, elem, node); + free(elem); + } +} + /* * Pipeline. */ @@ -1433,6 +2094,8 @@ rte_swx_pipeline_config(struct rte_swx_pipeline **p, int numa_node) TAILQ_INIT(&pipeline->extern_funcs); TAILQ_INIT(&pipeline->headers); TAILQ_INIT(&pipeline->actions); + TAILQ_INIT(&pipeline->table_types); + TAILQ_INIT(&pipeline->tables); pipeline->n_structs = 1; /* Struct 0 is reserved for action_data. */ pipeline->numa_node = numa_node; @@ -1447,6 +2110,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + table_state_free(p); + table_free(p); action_free(p); metadata_free(p); header_free(p); @@ -1499,10 +2164,20 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) if (status) goto error; + status = table_build(p); + if (status) + goto error; + + status = table_state_build(p); + if (status) + goto error; + p->build_done = 1; return 0; error: + table_state_build_free(p); + table_build_free(p); action_build_free(p); metadata_build_free(p); header_build_free(p); @@ -1514,3 +2189,28 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } + +/* + * Control. + */ +int +rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, + struct rte_swx_table_state **table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + *table_state = p->table_state; + return 0; +} + +int +rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, + struct rte_swx_table_state *table_state) +{ + if (!p || !table_state || !p->build_done) + return -EINVAL; + + p->table_state = table_state; + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 1b20293cb..d7e3ba1ec 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -19,6 +19,7 @@ extern "C" { #include <rte_compat.h> #include "rte_swx_port.h" +#include "rte_swx_table.h" #include "rte_swx_extern.h" /** Name size. */ @@ -377,6 +378,123 @@ rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char **instructions, uint32_t n_instructions); +/* + * Pipeline table + */ + +/** + * Pipeline table type register + * + * @param[in] p + * Pipeline handle. + * @param[in] name + * Table type name. + * @param[in] match type + * Match type implemented by the new table type. + * @param[in] ops + * Table type operations. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table type with this name already exists. + */ +__rte_experimental +int +rte_swx_pipeline_table_type_register(struct rte_swx_pipeline *p, + const char *name, + enum rte_swx_table_match_type match_type, + struct rte_swx_table_ops *ops); + +/** Match field parameters. */ +struct rte_swx_match_field_params { + /** Match field name. Must be either a field of one of the registered + * packet headers ("h.header.field") or a field of the registered + * meta-data ("m.field"). + */ + const char *name; + + /** Match type of the field. */ + enum rte_swx_table_match_type match_type; +}; + +/** Pipeline table parameters. */ +struct rte_swx_pipeline_table_params { + /** The set of match fields for the current table. + * Restriction: All the match fields of the current table need to be + * part of the same struct, i.e. either all the match fields are part of + * the same header or all the match fields are part of the meta-data. + */ + struct rte_swx_match_field_params *fields; + + /** The number of match fields for the current table. If set to zero, no + * "regular" entries (i.e. entries other than the default entry) can be + * added to the current table and the match process always results in + * lookup miss. + */ + uint32_t n_fields; + + /** The set of actions for the current table. */ + const char **action_names; + + /** The number of actions for the current table. Must be at least one. + */ + uint32_t n_actions; + + /** The default table action that gets executed on lookup miss. Must be + * one of the table actions included in the *action_names*. + */ + const char *default_action_name; + + /** Default action data. The size of this array is the action data size + * of the default action. Must be NULL if the default action data size + * is zero. + */ + uint8_t *default_action_data; + + /** If non-zero (true), then the default action of the current table + * cannot be changed. If zero (false), then the default action can be + * changed in the future with another action from the *action_names* + * list. + */ + int default_action_is_const; +}; + +/** + * Pipeline table configure + * + * @param[out] p + * Pipeline handle. + * @param[in] name + * Table name. + * @param[in] params + * Table parameters. + * @param[in] recommended_table_type_name + * Recommended table type. Typically set to NULL. Useful as guidance when + * there are multiple table types registered for the match type of the table, + * as determined from the table match fields specification. Silently ignored + * if the recommended table type does not exist or it serves a different match + * type. + * @param[in] args + * Table creation arguments. + * @param[in] size + * Guideline on maximum number of table entries. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory; + * -EEXIST: Table with this name already exists; + * -ENODEV: Table creation error. + */ +__rte_experimental +int +rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, + const char *name, + struct rte_swx_pipeline_table_params *params, + const char *recommended_table_type_name, + const char *args, + uint32_t size); + /** * Pipeline build * diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 6ad8a6b17..9df58698d 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -55,5 +55,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru_arm64.h endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index 71d134768..b9d4fe3dc 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -22,7 +22,8 @@ headers = files('rte_table.h', 'rte_table_hash_func_arm64.h', 'rte_lru.h', 'rte_table_array.h', - 'rte_table_stub.h') + 'rte_table_stub.h', + 'rte_swx_table.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table.h b/lib/librte_table/rte_swx_table.h new file mode 100644 index 000000000..c5c202723 --- /dev/null +++ b/lib/librte_table/rte_swx_table.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_H__ +#define __INCLUDE_RTE_SWX_TABLE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Table + * + * Table interface. + */ + +#include <stdint.h> +#include <sys/queue.h> + +/** Match type. */ +enum rte_swx_table_match_type { + /** Wildcard Match (WM). */ + RTE_SWX_TABLE_MATCH_WILDCARD, + + /** Longest Prefix Match (LPM). */ + RTE_SWX_TABLE_MATCH_LPM, + + /** Exact Match (EM). */ + RTE_SWX_TABLE_MATCH_EXACT, +}; + +/** Table creation parameters. */ +struct rte_swx_table_params { + /** Table match type. */ + enum rte_swx_table_match_type match_type; + + /** Key size in bytes. */ + uint32_t key_size; + + /** Offset of the first byte of the key within the key buffer. */ + uint32_t key_offset; + + /** Mask of *key_size* bytes logically laid over the bytes at positions + * *key_offset* .. (*key_offset* + *key_size* - 1) of the key buffer in + * order to specify which bits from the key buffer are part of the key + * and which ones are not. A bit value of 1 in the *key_mask0* means the + * respective bit in the key buffer is part of the key, while a bit + * value of 0 means the opposite. A NULL value means that all the bits + * are part of the key, i.e. the *key_mask0* is an all-ones mask. + */ + uint8_t *key_mask0; + + /** Maximum size (in bytes) of the action data. The data stored in the + * table for each entry is equal to *action_data_size* plus 8 bytes, + * which are used to store the action ID. + */ + uint32_t action_data_size; + + /** Maximum number of keys to be stored in the table together with their + * associated data. + */ + uint32_t n_keys_max; +}; + +/** Table entry. */ +struct rte_swx_table_entry { + /** Used to faciliate the addition of the current table entry to a + * linked list. + */ + TAILQ_ENTRY(rte_swx_table_entry) node; + + /** Key value for the current entry. Array of *key_size* bytes or NULL + * if the *key_size* for the current table is 0. + */ + uint8_t *key; + + /** Key mask for the current entry. Array of *key_size* bytes that is + * logically and'ed with *key_mask0* of the current table. A NULL value + * means that all the key bits already enabled by *key_mask0* are part + * of the key of the current entry. + */ + uint8_t *key_mask; + + /** Placeholder for a possible compressed version of the *key* and + * *key_mask* of the current entry. Typically a hash signature, its main + * purpose is to the linked list search operation. Should be ignored by + * the API functions below. + */ + uint64_t key_signature; + + /** Action ID for the current entry. */ + uint64_t action_id; + + /** Action data for the current entry. Its size is defined by the action + * specified by the *action_id*. It must be NULL when the action data + * size of the *action_id* action is NULL. It must never exceed the + * *action_data_size* of the table. + */ + uint8_t *action_data; +}; + +/** List of table entries. */ +TAILQ_HEAD(rte_swx_table_entry_list, rte_swx_table_entry); + +/** + * Table memory footprint get + * + * @param[in] params + * Table create parameters. + * @param[in] entries + * Table entries. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, if successful, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_footprint_get_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args); + +/** + * Table mailbox size get + * + * The mailbox is used to store the context of a lookup operation that is in + * progress and it is passed as a parameter to the lookup operation. This allows + * for multiple concurrent lookup operations into the same table. + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @return + * Table memory footprint in bytes, on success, or zero, on error. + */ +typedef uint64_t +(*rte_swx_table_mailbox_size_get_t)(void); + +/** + * Table create + * + * @param[in] params + * Table creation parameters. + * @param[in] entries + * Entries to be added to the table at creation time. + * @param[in] args + * Any additional table create arguments. It may be NULL. + * @param[in] numa_node + * Non-Uniform Memory Access (NUMA) node. + * @return + * Table handle, on success, or NULL, on error. + */ +typedef void * +(*rte_swx_table_create_t)(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node); + +/** + * Table entry add + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_add_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table entry delete + * + * @param[in] table + * Table handle. + * @param[in] entry + * Entry to be deleted from the table. The entry *action_id* and *action_data* + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid table handle, entry or entry field; + * -ENOSPC: Table full. + */ +typedef int +(*rte_swx_table_delete_t)(void *table, + struct rte_swx_table_entry *entry); + +/** + * Table lookup + * + * The table lookup operation seaches a given key in the table and upon its + * completion it returns an indication of whether the key is found in the table + * (lookup hit) or not (lookup miss). In case of lookup hit, the action_id and + * the action_data associated with the key are also returned. + * + * Multiple invocations of this function may be required in order to complete a + * single table lookup operation for a given table and a given lookup key. The + * completion of the table lookup operation is flagged by a return value of 1; + * in case of a return value of 0, the function must be invoked again with + * exactly the same arguments. + * + * The mailbox argument is used to store the context of an on-going table lookup + * operation. The mailbox mechanism allows for multiple concurrent table lookup + * operations into the same table. + * + * The typical reason an implementation may choose to split the table lookup + * operation into multiple steps is to hide the latency of the inherrent memory + * read operations: before a read operation with the source data likely not in + * the CPU cache, the source data prefetch is issued and the table lookup + * operation is postponed in favor of some other unrelated work, which the CPU + * executes in parallel with the source data being fetched into the CPU cache; + * later on, the table lookup operation is resumed, this time with the source + * data likely to be read from the CPU cache with no CPU pipeline stall, which + * significantly improves the table lookup performance. + * + * @param[in] table + * Table handle. + * @param[in] mailbox + * Mailbox for the current table lookup operation. + * @param[in] key + * Lookup key. Its size mult be equal to the table *key_size*. If the latter + * is zero, then the lookup key must be NULL. + * @param[out] action_id + * ID of the action associated with the *key*. Must point to a valid 64-bit + * variable. Only valid when the function returns 1 and *hit* is set to true. + * @param[out] action_data + * Action data for the *action_id* action. Must point to a valid array of + * table *action_data_size* bytes. Only valid when the function returns 1 and + * *hit* is set to true. + * @param[out] hit + * Only valid when the function returns 1. Set to non-zero (true) on table + * lookup hit and to zero (false) on table lookup miss. + * @return + * 0 when the table lookup operation is not yet completed, and 1 when the + * table lookup operation is completed. No other return values are allowed. + */ +typedef int +(*rte_swx_table_lookup_t)(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit); + +/** + * Table free + * + * @param[in] table + * Table handle. + */ +typedef void +(*rte_swx_table_free_t)(void *table); + +/** Table operations. */ +struct rte_swx_table_ops { + /** Table memory footprint get. Set to NULL when not supported. */ + rte_swx_table_footprint_get_t footprint_get; + + /** Table mailbox size get. When NULL, the mailbox size is 0. */ + rte_swx_table_mailbox_size_get_t mailbox_size_get; + + /** Table create. Must be non-NULL. */ + rte_swx_table_create_t create; + + /** Incremental table entry add. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the new entry included. + */ + rte_swx_table_add_t add; + + /** Incremental table entry delete. Set to NULL when not supported, in + * which case the existing table has to be destroyed and a new table + * built from scratch with the entry excluded. + */ + rte_swx_table_delete_t del; + + /** Table lookup. Must be non-NULL. */ + rte_swx_table_lookup_t lkp; + + /** Table free. Must be non-NULL. */ + rte_swx_table_free_t free; +}; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 08/40] pipeline: add pipeline instructions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (6 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 07/40] pipeline: add tables Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 09/40] pipeline: add rx and extract instructions Cristian Dumitrescu ` (31 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The pipeline instructions represent the main program that defines the life of the packet. As packets go through tables that trigger action subroutines, the headers and meta-data get transformed along the way. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 36 ++++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 20 +++++++++++ 3 files changed, 57 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index b9e59bce2..7139df0d3 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -70,6 +70,7 @@ EXPERIMENTAL { rte_swx_pipeline_action_config; rte_swx_pipeline_table_type_register; rte_swx_pipeline_table_config; + rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_table_state_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 43cdb0f7c..464f9faf8 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -274,6 +274,10 @@ struct thread { /* Extern objects and functions. */ struct extern_obj_runtime *extern_objs; struct extern_func_runtime *extern_funcs; + + /* Instructions. */ + struct instruction *ip; + struct instruction *ret; }; #ifndef RTE_SWX_PIPELINE_THREADS_MAX @@ -300,6 +304,7 @@ struct rte_swx_pipeline { struct port_out_runtime *out; struct instruction **action_instructions; struct rte_swx_table_state *table_state; + struct instruction *instructions; struct thread threads[RTE_SWX_PIPELINE_THREADS_MAX]; uint32_t n_structs; @@ -310,6 +315,7 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t n_instructions; int build_done; int numa_node; }; @@ -1424,6 +1430,12 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) +{ + t->ip = p->instructions; +} + static int instruction_config(struct rte_swx_pipeline *p __rte_unused, struct action *a __rte_unused, @@ -2110,6 +2122,8 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) if (!p) return; + free(p->instructions); + table_state_free(p); table_free(p); action_free(p); @@ -2124,6 +2138,28 @@ rte_swx_pipeline_free(struct rte_swx_pipeline *p) free(p); } +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions) +{ + int err; + uint32_t i; + + err = instruction_config(p, NULL, instructions, n_instructions); + if (err) + return err; + + /* Thread instruction pointer reset. */ + for (i = 0; i < RTE_SWX_PIPELINE_THREADS_MAX; i++) { + struct thread *t = &p->threads[i]; + + thread_ip_reset(p, t); + } + + return 0; +} + int rte_swx_pipeline_build(struct rte_swx_pipeline *p) { diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index d7e3ba1ec..47a0f8dcc 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -495,6 +495,26 @@ rte_swx_pipeline_table_config(struct rte_swx_pipeline *p, const char *args, uint32_t size); +/** + * Pipeline instructions configure + * + * @param[in] p + * Pipeline handle. + * @param[in] instructions + * Pipeline instructions. + * @param[in] n_instructions + * Number of pipeline instructions. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument; + * -ENOMEM: Not enough space/cannot allocate memory. + */ +__rte_experimental +int +rte_swx_pipeline_instructions_config(struct rte_swx_pipeline *p, + const char **instructions, + uint32_t n_instructions); + /** * Pipeline build * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 09/40] pipeline: add rx and extract instructions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (7 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 08/40] pipeline: add pipeline instructions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 10/40] pipeline: add tx and emit instructions Cristian Dumitrescu ` (30 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add packet reception and header extraction instructions. The RX must be the first pipeline instruction. Each extracted header is logically removed from the packet, then it can be read/written by instructions, emitted into the outgoing packet or discarded. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 564 ++++++++++++++++++- lib/librte_pipeline/rte_swx_pipeline.h | 13 + 3 files changed, 574 insertions(+), 4 deletions(-) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 7139df0d3..793957eb9 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -73,6 +73,7 @@ EXPERIMENTAL { rte_swx_pipeline_instructions_config; rte_swx_pipeline_build; rte_swx_pipeline_free; + rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; }; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 464f9faf8..98772de99 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -8,6 +8,7 @@ #include <sys/queue.h> #include <rte_common.h> +#include <rte_prefetch.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -21,6 +22,16 @@ do { \ #define CHECK_NAME(name, err_code) \ CHECK((name) && (name)[0], err_code) +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + /* * Struct. */ @@ -181,7 +192,64 @@ struct header_out_runtime { /* * Instruction. */ + +/* Packet headers are always in Network Byte Order (NBO), i.e. big endian. + * Packet meta-data fields are always assumed to be in Host Byte Order (HBO). + * Table entry fields can be in either NBO or HBO; they are assumed to be in HBO + * when transferred to packet meta-data and in NBO when transferred to packet + * headers. + */ + +/* Notation conventions: + * -Header field: H = h.header.field (dst/src) + * -Meta-data field: M = m.field (dst/src) + * -Extern object mailbox field: E = e.field (dst/src) + * -Extern function mailbox field: F = f.field (dst/src) + * -Table action data field: T = t.field (src only) + * -Immediate value: I = 32-bit unsigned value (src only) + */ + +enum instruction_type { + /* rx m.port_in */ + INSTR_RX, + + /* extract h.header */ + INSTR_HDR_EXTRACT, + INSTR_HDR_EXTRACT2, + INSTR_HDR_EXTRACT3, + INSTR_HDR_EXTRACT4, + INSTR_HDR_EXTRACT5, + INSTR_HDR_EXTRACT6, + INSTR_HDR_EXTRACT7, + INSTR_HDR_EXTRACT8, +}; + +struct instr_io { + struct { + uint8_t offset; + uint8_t n_bits; + uint8_t pad[2]; + } io; + + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + uint8_t n_bytes[8]; + } hdr; +}; + struct instruction { + enum instruction_type type; + union { + struct instr_io io; + }; +}; + +struct instruction_data { + char label[RTE_SWX_NAME_SIZE]; + char jmp_label[RTE_SWX_NAME_SIZE]; + uint32_t n_users; /* user = jmp instruction to this instruction. */ + int invalid; }; /* @@ -251,6 +319,10 @@ struct table_runtime { * Pipeline. */ struct thread { + /* Packet. */ + struct rte_swx_pkt pkt; + uint8_t *ptr; + /* Structures. */ uint8_t **structs; @@ -280,6 +352,29 @@ struct thread { struct instruction *ret; }; +#define MASK64_BIT_GET(mask, pos) ((mask) & (1LLU << (pos))) +#define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) +#define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) + +#define METADATA_READ(thread, offset, n_bits) \ +({ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + (m64 & m64_mask); \ +}) + +#define METADATA_WRITE(thread, offset, n_bits, value) \ +{ \ + uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ + uint64_t m64 = *m64_ptr; \ + uint64_t m64_mask = UINT64_MAX >> (64 - (n_bits)); \ + \ + uint64_t m_new = value; \ + \ + *m64_ptr = (m64 & ~m64_mask) | (m_new & m64_mask); \ +} + #ifndef RTE_SWX_PIPELINE_THREADS_MAX #define RTE_SWX_PIPELINE_THREADS_MAX 16 #endif @@ -315,6 +410,8 @@ struct rte_swx_pipeline { uint32_t n_actions; uint32_t n_tables; uint32_t n_headers; + uint32_t thread_id; + uint32_t port_id; uint32_t n_instructions; int build_done; int numa_node; @@ -1187,6 +1284,16 @@ header_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct header * +header_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'h' || name[1] != '.') + return NULL; + + return header_find(p, &name[2]); +} + static struct field * header_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1430,19 +1537,459 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static inline void +pipeline_port_inc(struct rte_swx_pipeline *p) +{ + p->port_id = (p->port_id + 1) & (p->n_ports_in - 1); +} + static inline void thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) { t->ip = p->instructions; } +static inline void +thread_ip_inc(struct rte_swx_pipeline *p); + +static inline void +thread_ip_inc(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + t->ip++; +} + +static inline void +thread_ip_inc_cond(struct thread *t, int cond) +{ + t->ip += cond; +} + +static inline void +thread_yield(struct rte_swx_pipeline *p) +{ + p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + +/* + * rx. + */ +static int +instr_rx_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_RX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_rx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct port_in_runtime *port = &p->in[p->port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + int pkt_received; + + /* Packet. */ + pkt_received = port->pkt_rx(port->obj, pkt); + t->ptr = &pkt->pkt[pkt->offset]; + rte_prefetch0(t->ptr); + + TRACE("[Thread %2u] rx %s from port %u\n", + p->thread_id, + pkt_received ? "1 pkt" : "0 pkts", + p->port_id); + + /* Headers. */ + t->valid_headers = 0; + t->n_headers_out = 0; + + /* Meta-data. */ + METADATA_WRITE(t, ip->io.io.offset, ip->io.io.n_bits, p->port_id); + + /* Tables. */ + t->table_state = p->table_state; + + /* Thread. */ + pipeline_port_inc(p); + thread_ip_inc_cond(t, pkt_received); + thread_yield(p); +} + +/* + * extract. + */ +static int +instr_hdr_extract_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EXTRACT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +instr_hdr_extract_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_extract_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_extract8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_extract_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + CHECK(0, EINVAL); +} + +static uint32_t +label_is_used(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t count = 0, i; + + if (!label[0]) + return 0; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].jmp_label)) + count++; + + return count; +} + static int -instruction_config(struct rte_swx_pipeline *p __rte_unused, - struct action *a __rte_unused, - const char **instructions __rte_unused, - uint32_t n_instructions __rte_unused) +instr_label_check(struct instruction_data *instruction_data, + uint32_t n_instructions) { + uint32_t i; + + /* Check that all instruction labels are unique. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + uint32_t j; + + if (!label[0]) + continue; + + for (j = i + 1; j < n_instructions; j++) + CHECK(strcmp(label, data[j].label), EINVAL); + } + + /* Get users for each instruction label. */ + for (i = 0; i < n_instructions; i++) { + struct instruction_data *data = &instruction_data[i]; + char *label = data->label; + + data->n_users = label_is_used(instruction_data, + n_instructions, + label); + } + + return 0; +} + +static int +instruction_config(struct rte_swx_pipeline *p, + struct action *a, + const char **instructions, + uint32_t n_instructions) +{ + struct instruction *instr = NULL; + struct instruction_data *data = NULL; + char *string = NULL; + int err = 0; + uint32_t i; + + CHECK(n_instructions, EINVAL); + CHECK(instructions, EINVAL); + for (i = 0; i < n_instructions; i++) + CHECK(instructions[i], EINVAL); + + /* Memory allocation. */ + instr = calloc(n_instructions, sizeof(struct instruction)); + if (!instr) { + err = ENOMEM; + goto error; + } + + data = calloc(n_instructions, sizeof(struct instruction_data)); + if (!data) { + err = ENOMEM; + goto error; + } + + for (i = 0; i < n_instructions; i++) { + string = strdup(instructions[i]); + if (!string) { + err = ENOMEM; + goto error; + } + + err = instr_translate(p, a, string, &instr[i], &data[i]); + if (err) + goto error; + + free(string); + } + + err = instr_label_check(data, n_instructions); + if (err) + goto error; + + free(data); + + if (a) { + a->instructions = instr; + a->n_instructions = n_instructions; + } else { + p->instructions = instr; + p->n_instructions = n_instructions; + } + return 0; + +error: + free(string); + free(data); + free(instr); + return err; +} + +typedef void (*instr_exec_t)(struct rte_swx_pipeline *); + +static instr_exec_t instruction_table[] = { + [INSTR_RX] = instr_rx_exec, + + [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, + [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, + [INSTR_HDR_EXTRACT3] = instr_hdr_extract3_exec, + [INSTR_HDR_EXTRACT4] = instr_hdr_extract4_exec, + [INSTR_HDR_EXTRACT5] = instr_hdr_extract5_exec, + [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, + [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, + [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, +}; + +static inline void +instr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + instr_exec_t instr = instruction_table[ip->type]; + + instr(p); } /* @@ -2226,6 +2773,15 @@ rte_swx_pipeline_build(struct rte_swx_pipeline *p) return status; } +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) + instr_exec(p); +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 47a0f8dcc..fb83a8820 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -534,6 +534,19 @@ __rte_experimental int rte_swx_pipeline_build(struct rte_swx_pipeline *p); +/** + * Pipeline run + * + * @param[in] p + * Pipeline handle. + * @param[in] n_instructions + * Number of instructions to execute. + */ +__rte_experimental +void +rte_swx_pipeline_run(struct rte_swx_pipeline *p, + uint32_t n_instructions); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 10/40] pipeline: add tx and emit instructions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (8 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 09/40] pipeline: add rx and extract instructions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 11/40] pipeline: add header validate and invalidate instructions Cristian Dumitrescu ` (29 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add header emit and packet transmission instructions. Emit adds to the output packet a header that is either generated (e.g. read from table entry by action) or extracted from the input packet. TX ends the pipeline processing; discard is implemented by tx to special port. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 328 +++++++++++++++++++++++++ 1 file changed, 328 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 98772de99..1a637068c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -213,6 +213,9 @@ enum instruction_type { /* rx m.port_in */ INSTR_RX, + /* tx m.port_out */ + INSTR_TX, + /* extract h.header */ INSTR_HDR_EXTRACT, INSTR_HDR_EXTRACT2, @@ -222,6 +225,17 @@ enum instruction_type { INSTR_HDR_EXTRACT6, INSTR_HDR_EXTRACT7, INSTR_HDR_EXTRACT8, + + /* emit h.header */ + INSTR_HDR_EMIT, + INSTR_HDR_EMIT_TX, + INSTR_HDR_EMIT2_TX, + INSTR_HDR_EMIT3_TX, + INSTR_HDR_EMIT4_TX, + INSTR_HDR_EMIT5_TX, + INSTR_HDR_EMIT6_TX, + INSTR_HDR_EMIT7_TX, + INSTR_HDR_EMIT8_TX, }; struct instr_io { @@ -1635,6 +1649,114 @@ instr_rx_exec(struct rte_swx_pipeline *p) thread_yield(p); } +/* + * tx. + */ +static int +instr_tx_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct field *f; + + CHECK(n_tokens == 2, EINVAL); + + f = metadata_field_parse(p, tokens[1]); + CHECK(f, EINVAL); + + instr->type = INSTR_TX; + instr->io.io.offset = f->offset / 8; + instr->io.io.n_bits = f->n_bits; + return 0; +} + +static inline void +emit_handler(struct thread *t) +{ + struct header_out_runtime *h0 = &t->headers_out[0]; + struct header_out_runtime *h1 = &t->headers_out[1]; + uint32_t offset = 0, i; + + /* No header change or header decapsulation. */ + if ((t->n_headers_out == 1) && + (h0->ptr + h0->n_bytes == t->ptr)) { + TRACE("Emit handler: no header change or header decap.\n"); + + t->pkt.offset -= h0->n_bytes; + t->pkt.length += h0->n_bytes; + + return; + } + + /* Header encapsulation (optionally, with prior header decasulation). */ + if ((t->n_headers_out == 2) && + (h1->ptr + h1->n_bytes == t->ptr) && + (h0->ptr == h0->ptr0)) { + uint32_t offset; + + TRACE("Emit handler: header encapsulation.\n"); + + offset = h0->n_bytes + h1->n_bytes; + memcpy(t->ptr - offset, h0->ptr, h0->n_bytes); + t->pkt.offset -= offset; + t->pkt.length += offset; + + return; + } + + /* Header insertion. */ + /* TBD */ + + /* Header extraction. */ + /* TBD */ + + /* For any other case. */ + TRACE("Emit handler: complex case.\n"); + + for (i = 0; i < t->n_headers_out; i++) { + struct header_out_runtime *h = &t->headers_out[i]; + + memcpy(&t->header_out_storage[offset], h->ptr, h->n_bytes); + offset += h->n_bytes; + } + + if (offset) { + memcpy(t->ptr - offset, t->header_out_storage, offset); + t->pkt.offset -= offset; + t->pkt.length += offset; + } +} + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p); + +static inline void +instr_tx_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint64_t port_id = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + struct port_out_runtime *port = &p->out[port_id]; + struct rte_swx_pkt *pkt = &t->pkt; + + TRACE("[Thread %2u]: tx 1 pkt to port %u\n", + p->thread_id, + (uint32_t)port_id); + + /* Headers. */ + emit_handler(t); + + /* Packet. */ + port->pkt_tx(port->obj, pkt); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); +} + /* * extract. */ @@ -1797,6 +1919,185 @@ instr_hdr_extract8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * emit. + */ +static int +instr_hdr_emit_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_EMIT; + instr->io.hdr.header_id[0] = h->id; + instr->io.hdr.struct_id[0] = h->struct_id; + instr->io.hdr.n_bytes[0] = h->st->n_bits / 8; + return 0; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr = t->structs[struct_id]; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (!i) { + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi->ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +instr_hdr_emit_exec(struct rte_swx_pipeline *p) +{ + __instr_hdr_emit_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 1); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 2); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 3); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 4); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 5); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 6); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 7); + instr_tx_exec(p); +} + +static inline void +instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", + p->thread_id); + + __instr_hdr_emit_exec(p, 8); + instr_tx_exec(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -1842,6 +2143,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + if (!strcmp(tokens[tpos], "extract")) return instr_hdr_extract_translate(p, action, @@ -1850,6 +2159,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -1971,6 +2288,7 @@ typedef void (*instr_exec_t)(struct rte_swx_pipeline *); static instr_exec_t instruction_table[] = { [INSTR_RX] = instr_rx_exec, + [INSTR_TX] = instr_tx_exec, [INSTR_HDR_EXTRACT] = instr_hdr_extract_exec, [INSTR_HDR_EXTRACT2] = instr_hdr_extract2_exec, @@ -1980,6 +2298,16 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EXTRACT6] = instr_hdr_extract6_exec, [INSTR_HDR_EXTRACT7] = instr_hdr_extract7_exec, [INSTR_HDR_EXTRACT8] = instr_hdr_extract8_exec, + + [INSTR_HDR_EMIT] = instr_hdr_emit_exec, + [INSTR_HDR_EMIT_TX] = instr_hdr_emit_tx_exec, + [INSTR_HDR_EMIT2_TX] = instr_hdr_emit2_tx_exec, + [INSTR_HDR_EMIT3_TX] = instr_hdr_emit3_tx_exec, + [INSTR_HDR_EMIT4_TX] = instr_hdr_emit4_tx_exec, + [INSTR_HDR_EMIT5_TX] = instr_hdr_emit5_tx_exec, + [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, + [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, + [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 11/40] pipeline: add header validate and invalidate instructions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (9 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 10/40] pipeline: add tx and emit instructions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 12/40] pipeline: add mov instruction Cristian Dumitrescu ` (28 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add instructions to flag a header as valid or invalid. This flag can be tested by the jmpv (jump if header valid) and jmpnv (jump if header not valid) instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 108 +++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 1a637068c..8b1f290c0 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -236,6 +236,12 @@ enum instruction_type { INSTR_HDR_EMIT6_TX, INSTR_HDR_EMIT7_TX, INSTR_HDR_EMIT8_TX, + + /* validate h.header */ + INSTR_HDR_VALIDATE, + + /* invalidate h.header */ + INSTR_HDR_INVALIDATE, }; struct instr_io { @@ -252,10 +258,15 @@ struct instr_io { } hdr; }; +struct instr_hdr_validity { + uint8_t header_id; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; + struct instr_hdr_validity valid; }; }; @@ -2098,6 +2109,84 @@ instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) instr_tx_exec(p); } +/* + * validate. + */ +static int +instr_hdr_validate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_VALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_validate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] validate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_SET(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + +/* + * invalidate. + */ +static int +instr_hdr_invalidate_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct header *h; + + CHECK(n_tokens == 2, EINVAL); + + h = header_parse(p, tokens[1]); + CHECK(h, EINVAL); + + instr->type = INSTR_HDR_INVALIDATE; + instr->valid.header_id = h->id; + return 0; +} + +static inline void +instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->valid.header_id; + + TRACE("[Thread %2u] invalidate header %u\n", p->thread_id, header_id); + + /* Headers. */ + t->valid_headers = MASK64_BIT_CLR(t->valid_headers, header_id); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2167,6 +2256,22 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2308,6 +2413,9 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_EMIT6_TX] = instr_hdr_emit6_tx_exec, [INSTR_HDR_EMIT7_TX] = instr_hdr_emit7_tx_exec, [INSTR_HDR_EMIT8_TX] = instr_hdr_emit8_tx_exec, + + [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, + [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 12/40] pipeline: add mov instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (10 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 11/40] pipeline: add header validate and invalidate instructions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 13/40] pipeline: add dma instruction Cristian Dumitrescu ` (27 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The mov (i.e. move) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 369 +++++++++++++++++++++++++ 1 file changed, 369 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8b1f290c0..8963c1831 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6,9 +6,11 @@ #include <stdio.h> #include <errno.h> #include <sys/queue.h> +#include <arpa/inet.h> #include <rte_common.h> #include <rte_prefetch.h> +#include <rte_byteorder.h> #include "rte_swx_pipeline.h" #include "rte_swx_ctl.h" @@ -32,6 +34,9 @@ do { \ #define TRACE(...) #endif +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + /* * Struct. */ @@ -242,6 +247,21 @@ enum instruction_type { /* invalidate h.header */ INSTR_HDR_INVALIDATE, + + /* mov dst src + * dst = src + * dst = HMEF, src = HMEFTI + */ + INSTR_MOV, /* dst = MEF, src = MEFT */ + INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_MOV_I, /* dst = HMEF, src = I */ +}; + +struct instr_operand { + uint8_t struct_id; + uint8_t n_bits; + uint8_t offset; + uint8_t pad; }; struct instr_io { @@ -262,11 +282,20 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_dst_src { + struct instr_operand dst; + union { + struct instr_operand src; + uint32_t src_val; + }; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; + struct instr_dst_src mov; }; }; @@ -381,6 +410,57 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define MOV(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->mov.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define MOV_S(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->mov.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->mov.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->mov.src.n_bits); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + +#else + +#define MOV_S MOV + +#endif + +#define MOV_I(thread, ip) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->mov.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->mov.dst.n_bits); \ + \ + uint64_t src = (ip)->mov.src_val; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ +} + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -944,6 +1024,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **object) +{ + struct extern_obj *obj; + struct field *f; + char *obj_name, *field_name; + + if ((name[0] != 'e') || (name[1] != '.')) + return NULL; + + obj_name = strdup(&name[2]); + if (!obj_name) + return NULL; + + field_name = strchr(obj_name, '.'); + if (!field_name) { + free(obj_name); + return NULL; + } + + *field_name = 0; + field_name++; + + obj = extern_obj_find(p, obj_name); + if (!obj) { + free(obj_name); + return NULL; + } + + f = struct_type_field_find(obj->type->mailbox_struct_type, field_name); + if (!f) { + free(obj_name); + return NULL; + } + + if (object) + *object = obj; + + free(obj_name); + return f; +} + int rte_swx_pipeline_extern_type_register(struct rte_swx_pipeline *p, const char *name, @@ -1182,6 +1306,50 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_func **function) +{ + struct extern_func *func; + struct field *f; + char *func_name, *field_name; + + if ((name[0] != 'f') || (name[1] != '.')) + return NULL; + + func_name = strdup(&name[2]); + if (!func_name) + return NULL; + + field_name = strchr(func_name, '.'); + if (!field_name) { + free(func_name); + return NULL; + } + + *field_name = 0; + field_name++; + + func = extern_func_find(p, func_name); + if (!func) { + free(func_name); + return NULL; + } + + f = struct_type_field_find(func->mailbox_struct_type, field_name); + if (!f) { + free(func_name); + return NULL; + } + + if (function) + *function = func; + + free(func_name); + return f; +} + int rte_swx_pipeline_extern_func_register(struct rte_swx_pipeline *p, const char *name, @@ -1562,6 +1730,82 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static struct field * +action_field_parse(struct action *action, const char *name); + +static struct field * +struct_field_parse(struct rte_swx_pipeline *p, + struct action *action, + const char *name, + uint32_t *struct_id) +{ + struct field *f; + + switch (name[0]) { + case 'h': + { + struct header *header; + + f = header_field_parse(p, name, &header); + if (!f) + return NULL; + + *struct_id = header->struct_id; + return f; + } + + case 'm': + { + f = metadata_field_parse(p, name); + if (!f) + return NULL; + + *struct_id = p->metadata_struct_id; + return f; + } + + case 't': + { + if (!action) + return NULL; + + f = action_field_parse(action, name); + if (!f) + return NULL; + + *struct_id = 0; + return f; + } + + case 'e': + { + struct extern_obj *obj; + + f = extern_obj_mailbox_field_parse(p, name, &obj); + if (!f) + return NULL; + + *struct_id = obj->struct_id; + return f; + } + + case 'f': + { + struct extern_func *func; + + f = extern_func_mailbox_field_parse(p, name, &func); + if (!f) + return NULL; + + *struct_id = func->struct_id; + return f; + } + + default: + return NULL; + } +} + static inline void pipeline_port_inc(struct rte_swx_pipeline *p) { @@ -2187,6 +2431,104 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * mov. + */ +static int +instr_mov_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* MOV or MOV_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_MOV; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_MOV_S; + + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src.struct_id = (uint8_t)src_struct_id; + instr->mov.src.n_bits = fsrc->n_bits; + instr->mov.src.offset = fsrc->offset / 8; + return 0; + } + + /* MOV_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_MOV_I; + instr->mov.dst.struct_id = (uint8_t)dst_struct_id; + instr->mov.dst.n_bits = fdst->n_bits; + instr->mov.dst.offset = fdst->offset / 8; + instr->mov.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_mov_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov\n", + p->thread_id); + + MOV(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov (s)\n", + p->thread_id); + + MOV_S(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_mov_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] mov m.f %x\n", + p->thread_id, + ip->mov.src_val); + + MOV_I(t, ip); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2272,6 +2614,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2416,6 +2766,10 @@ static instr_exec_t instruction_table[] = { [INSTR_HDR_VALIDATE] = instr_hdr_validate_exec, [INSTR_HDR_INVALIDATE] = instr_hdr_invalidate_exec, + + [INSTR_MOV] = instr_mov_exec, + [INSTR_MOV_S] = instr_mov_s_exec, + [INSTR_MOV_I] = instr_mov_i_exec, }; static inline void @@ -2446,6 +2800,21 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct field * +action_field_find(struct action *a, const char *name) +{ + return a->st ? struct_type_field_find(a->st, name) : NULL; +} + +static struct field * +action_field_parse(struct action *action, const char *name) +{ + if (name[0] != 't' || name[1] != '.') + return NULL; + + return action_field_find(action, &name[2]); +} + int rte_swx_pipeline_action_config(struct rte_swx_pipeline *p, const char *name, -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 13/40] pipeline: add dma instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (11 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 12/40] pipeline: add mov instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 14/40] pipeline: introduce add instruction Cristian Dumitrescu ` (26 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The DMA instruction handles the bulk read transfer of one header from the table entry action data. Typically used to generate headers, i.e. headers that are not extracted from the input packet. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 207 +++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 8963c1831..07b9f7ef4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -255,6 +255,18 @@ enum instruction_type { INSTR_MOV, /* dst = MEF, src = MEFT */ INSTR_MOV_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_MOV_I, /* dst = HMEF, src = I */ + + /* dma h.header t.field + * memcpy(h.header, t.field, sizeof(h.header)) + */ + INSTR_DMA_HT, + INSTR_DMA_HT2, + INSTR_DMA_HT3, + INSTR_DMA_HT4, + INSTR_DMA_HT5, + INSTR_DMA_HT6, + INSTR_DMA_HT7, + INSTR_DMA_HT8, }; struct instr_operand { @@ -290,12 +302,26 @@ struct instr_dst_src { }; }; +struct instr_dma { + struct { + uint8_t header_id[8]; + uint8_t struct_id[8]; + } dst; + + struct { + uint8_t offset[8]; + } src; + + uint16_t n_bytes[8]; +}; + struct instruction { enum instruction_type type; union { struct instr_io io; struct instr_hdr_validity valid; struct instr_dst_src mov; + struct instr_dma dma; }; }; @@ -2529,6 +2555,170 @@ instr_mov_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * dma. + */ +static int +instr_dma_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1]; + char *src = tokens[2]; + struct header *h; + struct field *tf; + + CHECK(action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + h = header_parse(p, dst); + CHECK(h, EINVAL); + + tf = action_field_parse(action, src); + CHECK(tf, EINVAL); + + instr->type = INSTR_DMA_HT; + instr->dma.dst.header_id[0] = h->id; + instr->dma.dst.struct_id[0] = h->struct_id; + instr->dma.n_bytes[0] = h->st->n_bits / 8; + instr->dma.src.offset[0] = tf->offset / 8; + + return 0; +} + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma); + +static inline void +__instr_dma_ht_exec(struct rte_swx_pipeline *p, uint32_t n_dma) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *action_data = t->structs[0]; + uint64_t valid_headers = t->valid_headers; + uint32_t i; + + for (i = 0; i < n_dma; i++) { + uint32_t header_id = ip->dma.dst.header_id[i]; + uint32_t struct_id = ip->dma.dst.struct_id[i]; + uint32_t offset = ip->dma.src.offset[i]; + uint32_t n_bytes = ip->dma.n_bytes[i]; + + struct header_runtime *h = &t->headers[header_id]; + uint8_t *h_ptr0 = h->ptr0; + uint8_t *h_ptr = t->structs[struct_id]; + + void *dst = MASK64_BIT_GET(valid_headers, header_id) ? + h_ptr : h_ptr0; + void *src = &action_data[offset]; + + TRACE("[Thread %2u] dma h.s t.f\n", p->thread_id); + + /* Headers. */ + memcpy(dst, src, n_bytes); + t->structs[struct_id] = dst; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + } + + t->valid_headers = valid_headers; +} + +static inline void +instr_dma_ht_exec(struct rte_swx_pipeline *p) +{ + __instr_dma_ht_exec(p, 1); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht2_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 2); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht3_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 3); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht4_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 4); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht5_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 5); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht6_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 6); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht7_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 7); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_dma_ht8_exec(struct rte_swx_pipeline *p) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", + p->thread_id); + + __instr_dma_ht_exec(p, 8); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2622,6 +2812,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2770,6 +2968,15 @@ static instr_exec_t instruction_table[] = { [INSTR_MOV] = instr_mov_exec, [INSTR_MOV_S] = instr_mov_s_exec, [INSTR_MOV_I] = instr_mov_i_exec, + + [INSTR_DMA_HT] = instr_dma_ht_exec, + [INSTR_DMA_HT2] = instr_dma_ht2_exec, + [INSTR_DMA_HT3] = instr_dma_ht3_exec, + [INSTR_DMA_HT4] = instr_dma_ht4_exec, + [INSTR_DMA_HT5] = instr_dma_ht5_exec, + [INSTR_DMA_HT6] = instr_dma_ht6_exec, + [INSTR_DMA_HT7] = instr_dma_ht7_exec, + [INSTR_DMA_HT8] = instr_dma_ht8_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 14/40] pipeline: introduce add instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (12 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 13/40] pipeline: add dma instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 15/40] pipeline: introduce sub instruction Cristian Dumitrescu ` (25 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The add instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 302 +++++++++++++++++++++++++ 1 file changed, 302 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 07b9f7ef4..f7569ad6f 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -267,6 +267,17 @@ enum instruction_type { INSTR_DMA_HT6, INSTR_DMA_HT7, INSTR_DMA_HT8, + + /* add dst src + * dst += src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_ADD, /* dst = MEF, src = MEF */ + INSTR_ALU_ADD_MH, /* dst = MEF, src = H */ + INSTR_ALU_ADD_HM, /* dst = H, src = MEF */ + INSTR_ALU_ADD_HH, /* dst = H, src = H */ + INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ + INSTR_ALU_ADD_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -322,6 +333,7 @@ struct instruction { struct instr_hdr_validity valid; struct instr_dst_src mov; struct instr_dma dma; + struct instr_dst_src alu; }; }; @@ -436,6 +448,136 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define ALU(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_S(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MH ALU_S + +#define ALU_HM(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src64_mask = UINT64_MAX >> (64 - (ip)->alu.src.n_bits); \ + uint64_t src = src64 & src64_mask; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#define ALU_HH(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint8_t *src_struct = (thread)->structs[(ip)->alu.src.struct_id]; \ + uint64_t *src64_ptr = (uint64_t *)&src_struct[(ip)->alu.src.offset]; \ + uint64_t src64 = *src64_ptr; \ + uint64_t src = ntoh64(src64) >> (64 - (ip)->alu.src.n_bits); \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_S ALU +#define ALU_MH ALU +#define ALU_HM ALU +#define ALU_HH ALU + +#endif + +#define ALU_I(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = dst64 & dst64_mask; \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | (result & dst64_mask); \ +} + +#define ALU_MI ALU_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define ALU_HI(thread, ip, operator) \ +{ \ + uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ + uint64_t *dst64_ptr = (uint64_t *)&dst_struct[(ip)->alu.dst.offset]; \ + uint64_t dst64 = *dst64_ptr; \ + uint64_t dst64_mask = UINT64_MAX >> (64 - (ip)->alu.dst.n_bits); \ + uint64_t dst = ntoh64(dst64) >> (64 - (ip)->alu.dst.n_bits); \ + \ + uint64_t src = (ip)->alu.src_val; \ + \ + uint64_t result = dst operator src; \ + result = hton64(result << (64 - (ip)->alu.dst.n_bits)); \ + \ + *dst64_ptr = (dst64 & ~dst64_mask) | result; \ +} + +#else + +#define ALU_HI ALU_I + +#endif + #define MOV(thread, ip) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->mov.dst.struct_id]; \ @@ -2719,6 +2861,151 @@ instr_dma_ht8_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * alu. + */ +static int +instr_alu_add_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* ADD, ADD_HM, ADD_MH, ADD_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_ADD; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_ADD_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_ADD_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* ADD_MI, ADD_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_ADD_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_ADD_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + +static inline void +instr_alu_add_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_add_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] add (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, +); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -2820,6 +3107,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -2977,6 +3272,13 @@ static instr_exec_t instruction_table[] = { [INSTR_DMA_HT6] = instr_dma_ht6_exec, [INSTR_DMA_HT7] = instr_dma_ht7_exec, [INSTR_DMA_HT8] = instr_dma_ht8_exec, + + [INSTR_ALU_ADD] = instr_alu_add_exec, + [INSTR_ALU_ADD_MH] = instr_alu_add_mh_exec, + [INSTR_ALU_ADD_HM] = instr_alu_add_hm_exec, + [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, + [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, + [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 15/40] pipeline: introduce sub instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (13 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 14/40] pipeline: introduce add instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 16/40] pipeline: introduce ckadd instruction Cristian Dumitrescu ` (24 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The sub (i.e. subtract) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index f7569ad6f..79629a15e 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -278,6 +278,17 @@ enum instruction_type { INSTR_ALU_ADD_HH, /* dst = H, src = H */ INSTR_ALU_ADD_MI, /* dst = MEF, src = I */ INSTR_ALU_ADD_HI, /* dst = H, src = I */ + + /* sub dst src + * dst -= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SUB, /* dst = MEF, src = MEF */ + INSTR_ALU_SUB_MH, /* dst = MEF, src = H */ + INSTR_ALU_SUB_HM, /* dst = H, src = MEF */ + INSTR_ALU_SUB_HH, /* dst = H, src = H */ + INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ + INSTR_ALU_SUB_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -2916,6 +2927,58 @@ instr_alu_add_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_sub_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SUB, SUB_HM, SUB_MH, SUB_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SUB; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SUB_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SUB_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SUB_MI, SUB_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SUB_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SUB_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3006,6 +3069,96 @@ instr_alu_add_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_sub_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] sub (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, -); + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3115,6 +3268,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3279,6 +3440,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_ADD_HH] = instr_alu_add_hh_exec, [INSTR_ALU_ADD_MI] = instr_alu_add_mi_exec, [INSTR_ALU_ADD_HI] = instr_alu_add_hi_exec, + + [INSTR_ALU_SUB] = instr_alu_sub_exec, + [INSTR_ALU_SUB_MH] = instr_alu_sub_mh_exec, + [INSTR_ALU_SUB_HM] = instr_alu_sub_hm_exec, + [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, + [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, + [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 16/40] pipeline: introduce ckadd instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (14 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 15/40] pipeline: introduce sub instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 17/40] pipeline: introduce cksub instruction Cristian Dumitrescu ` (23 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The ckadd (i.e. checksum add) instruction is used to either compute, verify or update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 230 +++++++++++++++++++++++++ 1 file changed, 230 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 79629a15e..be5758a4a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -289,6 +289,14 @@ enum instruction_type { INSTR_ALU_SUB_HH, /* dst = H, src = H */ INSTR_ALU_SUB_MI, /* dst = MEF, src = I */ INSTR_ALU_SUB_HI, /* dst = H, src = I */ + + /* ckadd dst src + * dst = dst '+ src[0:1] '+ src[2:3] + ... + * dst = H, src = {H, h.header} + */ + INSTR_ALU_CKADD_FIELD, /* src = H */ + INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ + INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ }; struct instr_operand { @@ -2979,6 +2987,53 @@ instr_alu_sub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_ckadd_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + /* CKADD_FIELD. */ + fsrc = header_field_parse(p, src, &hsrc); + if (fsrc) { + instr->type = INSTR_ALU_CKADD_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* CKADD_STRUCT, CKADD_STRUCT20. */ + hsrc = header_parse(p, src); + CHECK(hsrc, EINVAL); + + instr->type = INSTR_ALU_CKADD_STRUCT; + if ((hsrc->st->n_bits / 8) == 20) + instr->type = INSTR_ALU_CKADD_STRUCT20; + + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = hsrc->st->n_bits; + instr->alu.src.offset = 0; /* Unused. */ + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3159,6 +3214,169 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] ckadd (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* The first input (r) is a 16-bit number. The second and the third + * inputs are 32-bit numbers. In the worst case scenario, the sum of the + * three numbers (output r) is a 34-bit number. + */ + r += (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is an 18-bit + * number. In the worst case scenario, the sum of the two numbers is a + * 19-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 3-bit number (0 .. 7). Their sum is a 17-bit number (0 .. 0x10006). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x10006), the output r is (0 .. 7). So no carry bit can be generated, + * therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r0, r1; + + TRACE("[Thread %2u] ckadd (struct of 20 bytes)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + r0 = src32_ptr[0]; /* r0 is a 32-bit number. */ + r1 = src32_ptr[1]; /* r1 is a 32-bit number. */ + r0 += src32_ptr[2]; /* The output r0 is a 33-bit number. */ + r1 += src32_ptr[3]; /* The output r1 is a 33-bit number. */ + r0 += r1 + src32_ptr[4]; /* The output r0 is a 35-bit number. */ + + /* The first input is a 16-bit number. The second input is a 19-bit + * number. Their sum is a 20-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 4-bit number (0 .. 15). The sum is a 17-bit number (0 .. 0x1000E). + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1000E), the output r is (0 .. 15). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r0 = (r0 & 0xFFFF) + (r0 >> 16); + + r0 = ~r0 & 0xFFFF; + r0 = r0 ? r0 : 0xFFFF; + + *dst16_ptr = (uint16_t)r0; + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr; + uint32_t *src32_ptr; + uint64_t r = 0; + uint32_t i; + + TRACE("[Thread %2u] ckadd (struct)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + + src_struct = t->structs[ip->alu.src.struct_id]; + src32_ptr = (uint32_t *)&src_struct[0]; + + /* The max number of 32-bit words in a 256-byte header is 8 = 2^3. + * Therefore, in the worst case scenario, a 35-bit number is added to a + * 16-bit number (the input r), so the output r is 36-bit number. + */ + for (i = 0; i < ip->alu.src.n_bits / 32; i++, src32_ptr++) + r += *src32_ptr; + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1000E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + #define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 static int @@ -3276,6 +3494,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3447,6 +3673,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SUB_HH] = instr_alu_sub_hh_exec, [INSTR_ALU_SUB_MI] = instr_alu_sub_mi_exec, [INSTR_ALU_SUB_HI] = instr_alu_sub_hi_exec, + + [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, + [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, + [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 17/40] pipeline: introduce cksub instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (15 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 16/40] pipeline: introduce ckadd instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 18/40] pipeline: introduce and instruction Cristian Dumitrescu ` (22 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The cksub (i.e. checksum subtract) instruction is used to update the 1's complement sum commonly used by protocols such as IPv4, TCP or UDP. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 116 +++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index be5758a4a..bb58aea7f 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -297,6 +297,12 @@ enum instruction_type { INSTR_ALU_CKADD_FIELD, /* src = H */ INSTR_ALU_CKADD_STRUCT20, /* src = h.header, with sizeof(header) = 20 */ INSTR_ALU_CKADD_STRUCT, /* src = h.hdeader, with any sizeof(header) */ + + /* cksub dst src + * dst = dst '- src + * dst = H, src = H + */ + INSTR_ALU_CKSUB_FIELD, }; struct instr_operand { @@ -3034,6 +3040,36 @@ instr_alu_ckadd_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_cksub_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct header *hdst, *hsrc; + struct field *fdst, *fsrc; + + CHECK(n_tokens == 3, EINVAL); + + fdst = header_field_parse(p, dst, &hdst); + CHECK(fdst && (fdst->n_bits == 16), EINVAL); + + fsrc = header_field_parse(p, src, &hsrc); + CHECK(fsrc, EINVAL); + + instr->type = INSTR_ALU_CKSUB_FIELD; + instr->alu.dst.struct_id = (uint8_t)hdst->struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)hsrc->struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3273,6 +3309,77 @@ instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_cksub_field_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint8_t *dst_struct, *src_struct; + uint16_t *dst16_ptr, dst; + uint64_t *src64_ptr, src64, src64_mask, src; + uint64_t r; + + TRACE("[Thread %2u] cksub (field)\n", p->thread_id); + + /* Structs. */ + dst_struct = t->structs[ip->alu.dst.struct_id]; + dst16_ptr = (uint16_t *)&dst_struct[ip->alu.dst.offset]; + dst = *dst16_ptr; + + src_struct = t->structs[ip->alu.src.struct_id]; + src64_ptr = (uint64_t *)&src_struct[ip->alu.src.offset]; + src64 = *src64_ptr; + src64_mask = UINT64_MAX >> (64 - ip->alu.src.n_bits); + src = src64 & src64_mask; + + r = dst; + r = ~r & 0xFFFF; + + /* Subtraction in 1's complement arithmetic (i.e. a '- b) is the same as + * the following sequence of operations in 2's complement arithmetic: + * a '- b = (a - b) % 0xFFFF. + * + * In order to prevent an underflow for the below subtraction, in which + * a 33-bit number (the subtrahend) is taken out of a 16-bit number (the + * minuend), we first add a multiple of the 0xFFFF modulus to the + * minuend. The number we add to the minuend needs to be a 34-bit number + * or higher, so for readability reasons we picked the 36-bit multiple. + * We are effectively turning the 16-bit minuend into a 36-bit number: + * (a - b) % 0xFFFF = (a + 0xFFFF00000 - b) % 0xFFFF. + */ + r += 0xFFFF00000ULL; /* The output r is a 36-bit number. */ + + /* A 33-bit number is subtracted from a 36-bit number (the input r). The + * result (the output r) is a 36-bit number. + */ + r -= (src >> 32) + (src & 0xFFFFFFFF); + + /* The first input is a 16-bit number. The second input is a 20-bit + * number. Their sum is a 21-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + /* The first input is a 16-bit number (0 .. 0xFFFF). The second input is + * a 5-bit number (0 .. 31). The sum is a 17-bit number (0 .. 0x1001E). + */ + r = (r & 0xFFFF) + (r >> 16); + + /* When the input r is (0 .. 0xFFFF), the output r is equal to the input + * r, so the output is (0 .. 0xFFFF). When the input r is (0x10000 .. + * 0x1001E), the output r is (0 .. 31). So no carry bit can be + * generated, therefore the output r is always a 16-bit number. + */ + r = (r & 0xFFFF) + (r >> 16); + + r = ~r & 0xFFFF; + r = r ? r : 0xFFFF; + + *dst16_ptr = (uint16_t)r; + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_struct20_exec(struct rte_swx_pipeline *p) { @@ -3502,6 +3609,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3677,6 +3792,7 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_FIELD] = instr_alu_ckadd_field_exec, [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, + [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 18/40] pipeline: introduce and instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (16 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 17/40] pipeline: introduce cksub instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 19/40] pipeline: introduce or instruction Cristian Dumitrescu ` (21 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The and (i.e. bitwise and) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index bb58aea7f..24c08ef67 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -303,6 +303,14 @@ enum instruction_type { * dst = H, src = H */ INSTR_ALU_CKSUB_FIELD, + + /* and dst src + * dst &= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_AND, /* dst = MEF, src = MEFT */ + INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_AND_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3070,6 +3078,55 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_and_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* AND or AND_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_AND; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_AND_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* AND_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_AND_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3250,6 +3307,51 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_and_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_and_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] and (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, &); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3617,6 +3719,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3793,6 +3903,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_CKADD_STRUCT] = instr_alu_ckadd_struct_exec, [INSTR_ALU_CKADD_STRUCT20] = instr_alu_ckadd_struct20_exec, [INSTR_ALU_CKSUB_FIELD] = instr_alu_cksub_field_exec, + + [INSTR_ALU_AND] = instr_alu_and_exec, + [INSTR_ALU_AND_S] = instr_alu_and_s_exec, + [INSTR_ALU_AND_I] = instr_alu_and_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 19/40] pipeline: introduce or instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (17 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 18/40] pipeline: introduce and instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 20/40] pipeline: introduce xor instruction Cristian Dumitrescu ` (20 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The or (i.e. bitwise or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 24c08ef67..317eedaad 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -311,6 +311,14 @@ enum instruction_type { INSTR_ALU_AND, /* dst = MEF, src = MEFT */ INSTR_ALU_AND_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_AND_I, /* dst = HMEF, src = I */ + + /* or dst src + * dst |= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_OR, /* dst = MEF, src = MEFT */ + INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_OR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3127,6 +3135,55 @@ instr_alu_and_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_or_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* OR or OR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_OR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_OR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* OR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_OR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3352,6 +3409,51 @@ instr_alu_and_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_or_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_or_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] or (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, |); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3727,6 +3829,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "or")) + return instr_alu_or_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -3907,6 +4017,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_AND] = instr_alu_and_exec, [INSTR_ALU_AND_S] = instr_alu_and_s_exec, [INSTR_ALU_AND_I] = instr_alu_and_i_exec, + + [INSTR_ALU_OR] = instr_alu_or_exec, + [INSTR_ALU_OR_S] = instr_alu_or_s_exec, + [INSTR_ALU_OR_I] = instr_alu_or_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 20/40] pipeline: introduce xor instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (18 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 19/40] pipeline: introduce or instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 21/40] pipeline: introduce shl instruction Cristian Dumitrescu ` (19 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The xor (i.e. bitwise exclusive or) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 114 +++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 317eedaad..20a831fb4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -319,6 +319,14 @@ enum instruction_type { INSTR_ALU_OR, /* dst = MEF, src = MEFT */ INSTR_ALU_OR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_OR_I, /* dst = HMEF, src = I */ + + /* xor dst src + * dst ^= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ + INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ + INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ }; struct instr_operand { @@ -3184,6 +3192,55 @@ instr_alu_or_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_xor_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* XOR or XOR_S. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_XOR; + if ((dst[0] == 'h' && src[0] != 'h') || + (dst[0] != 'h' && src[0] == 'h')) + instr->type = INSTR_ALU_XOR_S; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* XOR_I. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + if (dst[0] == 'h') + src_val = htonl(src_val); + + instr->type = INSTR_ALU_XOR_I; + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static inline void instr_alu_add_exec(struct rte_swx_pipeline *p) { @@ -3454,6 +3511,51 @@ instr_alu_or_i_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_xor_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (s)\n", p->thread_id); + + /* Structs. */ + ALU_S(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_xor_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] xor (i)\n", p->thread_id); + + /* Structs. */ + ALU_I(t, ip, ^); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_ckadd_field_exec(struct rte_swx_pipeline *p) { @@ -3837,6 +3939,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "xor")) + return instr_alu_xor_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4021,6 +4131,10 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_OR] = instr_alu_or_exec, [INSTR_ALU_OR_S] = instr_alu_or_s_exec, [INSTR_ALU_OR_I] = instr_alu_or_i_exec, + + [INSTR_ALU_XOR] = instr_alu_xor_exec, + [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, + [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 21/40] pipeline: introduce shl instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (19 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 20/40] pipeline: introduce xor instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 22/40] pipeline: introduce shr instruction Cristian Dumitrescu ` (18 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The shl (i.e. shift left) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 20a831fb4..c22bc007c 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -327,6 +327,17 @@ enum instruction_type { INSTR_ALU_XOR, /* dst = MEF, src = MEFT */ INSTR_ALU_XOR_S, /* (dst, src) = (MEF, H) or (dst, src) = (H, MEFT) */ INSTR_ALU_XOR_I, /* dst = HMEF, src = I */ + + /* shl dst src + * dst <<= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHL, /* dst = MEF, src = MEF */ + INSTR_ALU_SHL_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHL_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHL_HH, /* dst = H, src = H */ + INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHL_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3094,6 +3105,58 @@ instr_alu_cksub_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shl_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHL, SHL_HM, SHL_MH, SHL_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHL; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHL_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHL_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHL_MI, SHL_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHL_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHL_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3421,6 +3484,96 @@ instr_alu_sub_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shl_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shl (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, <<); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -3947,6 +4100,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shl")) + return instr_alu_shl_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4135,6 +4296,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_XOR] = instr_alu_xor_exec, [INSTR_ALU_XOR_S] = instr_alu_xor_s_exec, [INSTR_ALU_XOR_I] = instr_alu_xor_i_exec, + + [INSTR_ALU_SHL] = instr_alu_shl_exec, + [INSTR_ALU_SHL_MH] = instr_alu_shl_mh_exec, + [INSTR_ALU_SHL_HM] = instr_alu_shl_hm_exec, + [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, + [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, + [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 22/40] pipeline: introduce shr instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (20 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 21/40] pipeline: introduce shl instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 23/40] pipeline: introduce table instruction Cristian Dumitrescu ` (17 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The shr (i.e. shift right) instruction source can be header field (H), meta-data field (M), extern object (E) or function (F) mailbox field, table entry action data field (T) or immediate value (I). The destination is HMEF. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 168 +++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index c22bc007c..0c0490eef 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -338,6 +338,17 @@ enum instruction_type { INSTR_ALU_SHL_HH, /* dst = H, src = H */ INSTR_ALU_SHL_MI, /* dst = MEF, src = I */ INSTR_ALU_SHL_HI, /* dst = H, src = I */ + + /* shr dst src + * dst >>= src + * dst = HMEF, src = HMEFTI + */ + INSTR_ALU_SHR, /* dst = MEF, src = MEF */ + INSTR_ALU_SHR_MH, /* dst = MEF, src = H */ + INSTR_ALU_SHR_HM, /* dst = H, src = MEF */ + INSTR_ALU_SHR_HH, /* dst = H, src = H */ + INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ + INSTR_ALU_SHR_HI, /* dst = H, src = I */ }; struct instr_operand { @@ -3157,6 +3168,58 @@ instr_alu_shl_translate(struct rte_swx_pipeline *p, return 0; } +static int +instr_alu_shr_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *dst = tokens[1], *src = tokens[2]; + struct field *fdst, *fsrc; + uint32_t dst_struct_id, src_struct_id, src_val; + + CHECK(n_tokens == 3, EINVAL); + + fdst = struct_field_parse(p, NULL, dst, &dst_struct_id); + CHECK(fdst, EINVAL); + + /* SHR, SHR_HM, SHR_MH, SHR_HH. */ + fsrc = struct_field_parse(p, action, src, &src_struct_id); + if (fsrc) { + instr->type = INSTR_ALU_SHR; + if (dst[0] == 'h' && src[0] == 'm') + instr->type = INSTR_ALU_SHR_HM; + if (dst[0] == 'm' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_MH; + if (dst[0] == 'h' && src[0] == 'h') + instr->type = INSTR_ALU_SHR_HH; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src.struct_id = (uint8_t)src_struct_id; + instr->alu.src.n_bits = fsrc->n_bits; + instr->alu.src.offset = fsrc->offset / 8; + return 0; + } + + /* SHR_MI, SHR_HI. */ + src_val = strtoul(src, &src, 0); + CHECK(!src[0], EINVAL); + + instr->type = INSTR_ALU_SHR_MI; + if (dst[0] == 'h') + instr->type = INSTR_ALU_SHR_HI; + + instr->alu.dst.struct_id = (uint8_t)dst_struct_id; + instr->alu.dst.n_bits = fdst->n_bits; + instr->alu.dst.offset = fdst->offset / 8; + instr->alu.src_val = (uint32_t)src_val; + return 0; +} + static int instr_alu_and_translate(struct rte_swx_pipeline *p, struct action *action, @@ -3574,6 +3637,96 @@ instr_alu_shl_hi_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +static inline void +instr_alu_shr_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr\n", p->thread_id); + + /* Structs. */ + ALU(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mh)\n", p->thread_id); + + /* Structs. */ + ALU_MH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hm)\n", p->thread_id); + + /* Structs. */ + ALU_HM(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hh)\n", p->thread_id); + + /* Structs. */ + ALU_HH(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (mi)\n", p->thread_id); + + /* Structs. */ + ALU_MI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + +static inline void +instr_alu_shr_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] shr (hi)\n", p->thread_id); + + /* Structs. */ + ALU_HI(t, ip, >>); + + /* Thread. */ + thread_ip_inc(p); +} + static inline void instr_alu_and_exec(struct rte_swx_pipeline *p) { @@ -4108,6 +4261,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "shr")) + return instr_alu_shr_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4303,6 +4464,13 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHL_HH] = instr_alu_shl_hh_exec, [INSTR_ALU_SHL_MI] = instr_alu_shl_mi_exec, [INSTR_ALU_SHL_HI] = instr_alu_shl_hi_exec, + + [INSTR_ALU_SHR] = instr_alu_shr_exec, + [INSTR_ALU_SHR_MH] = instr_alu_shr_mh_exec, + [INSTR_ALU_SHR_HM] = instr_alu_shr_hm_exec, + [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, + [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, + [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 23/40] pipeline: introduce table instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (21 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 22/40] pipeline: introduce shr instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 24/40] pipeline: introduce extern instruction Cristian Dumitrescu ` (16 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The table instruction looks up the input key into the table and then it triggers the execution of the action found in the table entry. On lookup miss, the default table action is executed. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 100 +++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 0c0490eef..51741dc99 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -349,6 +349,9 @@ enum instruction_type { INSTR_ALU_SHR_HH, /* dst = H, src = H */ INSTR_ALU_SHR_MI, /* dst = MEF, src = I */ INSTR_ALU_SHR_HI, /* dst = H, src = I */ + + /* table TABLE */ + INSTR_TABLE, }; struct instr_operand { @@ -376,6 +379,10 @@ struct instr_hdr_validity { uint8_t header_id; }; +struct instr_table { + uint8_t table_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -405,6 +412,7 @@ struct instruction { struct instr_dst_src mov; struct instr_dma dma; struct instr_dst_src alu; + struct instr_table table; }; }; @@ -2057,6 +2065,15 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_action_call(struct rte_swx_pipeline *p, + struct thread *t, + uint32_t action_id) +{ + t->ret = t->ip + 1; + t->ip = p->action_instructions[action_id]; +} + static inline void thread_ip_inc(struct rte_swx_pipeline *p); @@ -2670,6 +2687,79 @@ instr_hdr_invalidate_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } +/* + * table. + */ +static struct table * +table_find(struct rte_swx_pipeline *p, const char *name); + +static int +instr_table_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + struct table *t; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); + + t = table_find(p, tokens[1]); + CHECK(t, EINVAL); + + instr->type = INSTR_TABLE; + instr->table.table_id = t->id; + return 0; +} + +static inline void +instr_table_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t table_id = ip->table.table_id; + struct rte_swx_table_state *ts = &t->table_state[table_id]; + struct table_runtime *table = &t->tables[table_id]; + uint64_t action_id; + uint8_t *action_data; + int done, hit; + + /* Table. */ + done = table->func(ts->obj, + table->mailbox, + table->key, + &action_id, + &action_data, + &hit); + if (!done) { + /* Thread. */ + TRACE("[Thread %2u] table %u (not finalized)\n", + p->thread_id, + table_id); + + thread_yield(p); + return; + } + + action_id = hit ? action_id : ts->default_action_id; + action_data = hit ? action_data : ts->default_action_data; + + TRACE("[Thread %2u] table %u (%s, action %u)\n", + p->thread_id, + table_id, + hit ? "hit" : "miss", + (uint32_t)action_id); + + t->action_id = action_id; + t->structs[0] = action_data; + t->hit = hit; + + /* Thread. */ + thread_ip_action_call(p, t, action_id); +} + /* * mov. */ @@ -4269,6 +4359,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "table")) + return instr_table_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4471,6 +4569,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HH] = instr_alu_shr_hh_exec, [INSTR_ALU_SHR_MI] = instr_alu_shr_mi_exec, [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, + + [INSTR_TABLE] = instr_table_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 24/40] pipeline: introduce extern instruction 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (22 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 23/40] pipeline: introduce table instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 25/40] pipeline: introduce jmp and return instructions Cristian Dumitrescu ` (15 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The extern instruction calls one of the member functions of a given extern object or it calls the given extern function. The function arguments must be written in advance in the maibox. The results are available in the same place after execution. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 175 +++++++++++++++++++++++++ 1 file changed, 175 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 51741dc99..1fb3a24af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -352,6 +352,12 @@ enum instruction_type { /* table TABLE */ INSTR_TABLE, + + /* extern e.obj.func */ + INSTR_EXTERN_OBJ, + + /* extern f.func */ + INSTR_EXTERN_FUNC, }; struct instr_operand { @@ -383,6 +389,15 @@ struct instr_table { uint8_t table_id; }; +struct instr_extern_obj { + uint8_t ext_obj_id; + uint8_t func_id; +}; + +struct instr_extern_func { + uint8_t ext_func_id; +}; + struct instr_dst_src { struct instr_operand dst; union { @@ -413,6 +428,8 @@ struct instruction { struct instr_dma dma; struct instr_dst_src alu; struct instr_table table; + struct instr_extern_obj ext_obj; + struct instr_extern_func ext_func; }; }; @@ -1271,6 +1288,50 @@ extern_obj_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_type_member_func * +extern_obj_member_func_parse(struct rte_swx_pipeline *p, + const char *name, + struct extern_obj **obj) +{ + struct extern_obj *object; + struct extern_type_member_func *func; + char *object_name, *func_name; + + if (name[0] != 'e' || name[1] != '.') + return NULL; + + object_name = strdup(&name[2]); + if (!object_name) + return NULL; + + func_name = strchr(object_name, '.'); + if (!func_name) { + free(object_name); + return NULL; + } + + *func_name = 0; + func_name++; + + object = extern_obj_find(p, object_name); + if (!object) { + free(object_name); + return NULL; + } + + func = extern_type_member_func_find(object->type, func_name); + if (!func) { + free(object_name); + return NULL; + } + + if (obj) + *obj = object; + + free(object_name); + return func; +} + static struct field * extern_obj_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -1553,6 +1614,16 @@ extern_func_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct extern_func * +extern_func_parse(struct rte_swx_pipeline *p, + const char *name) +{ + if (name[0] != 'f' || name[1] != '.') + return NULL; + + return extern_func_find(p, &name[2]); +} + static struct field * extern_func_mailbox_field_parse(struct rte_swx_pipeline *p, const char *name, @@ -2097,6 +2168,12 @@ thread_yield(struct rte_swx_pipeline *p) p->thread_id = (p->thread_id + 1) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); } +static inline void +thread_yield_cond(struct rte_swx_pipeline *p, int cond) +{ + p->thread_id = (p->thread_id + cond) & (RTE_SWX_PIPELINE_THREADS_MAX - 1); +} + /* * rx. */ @@ -2760,6 +2837,94 @@ instr_table_exec(struct rte_swx_pipeline *p) thread_ip_action_call(p, t, action_id); } +/* + * extern. + */ +static int +instr_extern_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + char *token = tokens[1]; + + CHECK(n_tokens == 2, EINVAL); + + if (token[0] == 'e') { + struct extern_obj *obj; + struct extern_type_member_func *func; + + func = extern_obj_member_func_parse(p, token, &obj); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_OBJ; + instr->ext_obj.ext_obj_id = obj->id; + instr->ext_obj.func_id = func->id; + + return 0; + } + + if (token[0] == 'f') { + struct extern_func *func; + + func = extern_func_parse(p, token); + CHECK(func, EINVAL); + + instr->type = INSTR_EXTERN_FUNC; + instr->ext_func.ext_func_id = func->id; + + return 0; + } + + CHECK(0, EINVAL); +} + +static inline void +instr_extern_obj_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t obj_id = ip->ext_obj.ext_obj_id; + uint32_t func_id = ip->ext_obj.func_id; + struct extern_obj_runtime *obj = &t->extern_objs[obj_id]; + rte_swx_extern_type_member_func_t func = obj->funcs[func_id]; + + TRACE("[Thread %2u] extern obj %u member func %u\n", + p->thread_id, + obj_id, + func_id); + + /* Extern object member function execute. */ + uint32_t done = func(obj->obj, obj->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + +static inline void +instr_extern_func_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t ext_func_id = ip->ext_func.ext_func_id; + struct extern_func_runtime *ext_func = &t->extern_funcs[ext_func_id]; + rte_swx_extern_func_t func = ext_func->func; + + TRACE("[Thread %2u] extern func %u\n", + p->thread_id, + ext_func_id); + + /* Extern function execute. */ + uint32_t done = func(ext_func->mailbox); + + /* Thread. */ + thread_ip_inc_cond(t, done); + thread_yield_cond(p, done ^ 1); +} + /* * mov. */ @@ -4367,6 +4532,14 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "extern")) + return instr_extern_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } @@ -4571,6 +4744,8 @@ static instr_exec_t instruction_table[] = { [INSTR_ALU_SHR_HI] = instr_alu_shr_hi_exec, [INSTR_TABLE] = instr_table_exec, + [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, + [INSTR_EXTERN_FUNC] = instr_extern_func_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 25/40] pipeline: introduce jmp and return instructions 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (23 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 24/40] pipeline: introduce extern instruction Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 26/40] pipeline: add instruction verifier Cristian Dumitrescu ` (14 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev The jump instructions are either unconditional (jmp) or conditional on positive/negative tests such as header validity (jmpv/jmpnv), table lookup hit/miss (jmph/jmpnh), executed action (jmpa/jmpna), equality (jmpeq/jmpneq), comparison result (jmplt/jmpgt). The return instruction resumes the pipeline execution after action subroutine. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 1323 ++++++++++++++++++++++-- 1 file changed, 1211 insertions(+), 112 deletions(-) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 1fb3a24af..4eb1f4228 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -358,6 +358,84 @@ enum instruction_type { /* extern f.func */ INSTR_EXTERN_FUNC, + + /* jmp LABEL + * Unconditional jump + */ + INSTR_JMP, + + /* jmpv LABEL h.header + * Jump if header is valid + */ + INSTR_JMP_VALID, + + /* jmpnv LABEL h.header + * Jump if header is invalid + */ + INSTR_JMP_INVALID, + + /* jmph LABEL + * Jump if table lookup hit + */ + INSTR_JMP_HIT, + + /* jmpnh LABEL + * Jump if table lookup miss + */ + INSTR_JMP_MISS, + + /* jmpa LABEL ACTION + * Jump if action run + */ + INSTR_JMP_ACTION_HIT, + + /* jmpna LABEL ACTION + * Jump if action not run + */ + INSTR_JMP_ACTION_MISS, + + /* jmpeq LABEL a b + * Jump is a is equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_EQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_EQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_EQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmpneq LABEL a b + * Jump is a is not equal to b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_NEQ, /* (a, b) = (MEFT, MEFT) or (a, b) = (H, H) */ + INSTR_JMP_NEQ_S, /* (a, b) = (MEFT, H) or (a, b) = (H, MEFT) */ + INSTR_JMP_NEQ_I, /* (a, b) = (MEFT, I) or (a, b) = (H, I) */ + + /* jmplt LABEL a b + * Jump if a is less than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_LT, /* a = MEF, b = MEF */ + INSTR_JMP_LT_MH, /* a = MEF, b = H */ + INSTR_JMP_LT_HM, /* a = H, b = MEF */ + INSTR_JMP_LT_HH, /* a = H, b = H */ + INSTR_JMP_LT_MI, /* a = MEF, b = I */ + INSTR_JMP_LT_HI, /* a = H, b = I */ + + /* jmpgt LABEL a b + * Jump if a is greater than b + * a = HMEFT, b = HMEFTI + */ + INSTR_JMP_GT, /* a = MEF, b = MEF */ + INSTR_JMP_GT_MH, /* a = MEF, b = H */ + INSTR_JMP_GT_HM, /* a = H, b = MEF */ + INSTR_JMP_GT_HH, /* a = H, b = H */ + INSTR_JMP_GT_MI, /* a = MEF, b = I */ + INSTR_JMP_GT_HI, /* a = H, b = I */ + + /* return + * Return from action + */ + INSTR_RETURN, }; struct instr_operand { @@ -419,6 +497,21 @@ struct instr_dma { uint16_t n_bytes[8]; }; +struct instr_jmp { + struct instruction *ip; + + union { + struct instr_operand a; + uint8_t header_id; + uint8_t action_id; + }; + + union { + struct instr_operand b; + uint32_t b_val; + }; +}; + struct instruction { enum instruction_type type; union { @@ -430,6 +523,7 @@ struct instruction { struct instr_table table; struct instr_extern_obj ext_obj; struct instr_extern_func ext_func; + struct instr_jmp jmp; }; }; @@ -544,6 +638,9 @@ struct thread { #define MASK64_BIT_SET(mask, pos) ((mask) | (1LLU << (pos))) #define MASK64_BIT_CLR(mask, pos) ((mask) & ~(1LLU << (pos))) +#define HEADER_VALID(thread, header_id) \ + MASK64_BIT_GET((thread)->valid_headers, header_id) + #define ALU(thread, ip, operator) \ { \ uint8_t *dst_struct = (thread)->structs[(ip)->alu.dst.struct_id]; \ @@ -725,6 +822,118 @@ struct thread { *dst64_ptr = (dst64 & ~dst64_mask) | (src & dst64_mask); \ } +#define JMP_CMP(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_S(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MH JMP_CMP_S + +#define JMP_CMP_HM(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b64_mask = UINT64_MAX >> (64 - (ip)->jmp.b.n_bits); \ + uint64_t b = b64 & b64_mask; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_HH(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint8_t *b_struct = (thread)->structs[(ip)->jmp.b.struct_id]; \ + uint64_t *b64_ptr = (uint64_t *)&b_struct[(ip)->jmp.b.offset]; \ + uint64_t b64 = *b64_ptr; \ + uint64_t b = ntoh64(b64) >> (64 - (ip)->jmp.b.n_bits); \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_S JMP_CMP +#define JMP_CMP_MH JMP_CMP +#define JMP_CMP_HM JMP_CMP +#define JMP_CMP_HH JMP_CMP + +#endif + +#define JMP_CMP_I(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a64_mask = UINT64_MAX >> (64 - (ip)->jmp.a.n_bits); \ + uint64_t a = a64 & a64_mask; \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#define JMP_CMP_MI JMP_CMP_I + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + +#define JMP_CMP_HI(thread, ip, operator) \ +{ \ + uint8_t *a_struct = (thread)->structs[(ip)->jmp.a.struct_id]; \ + uint64_t *a64_ptr = (uint64_t *)&a_struct[(ip)->jmp.a.offset]; \ + uint64_t a64 = *a64_ptr; \ + uint64_t a = ntoh64(a64) >> (64 - (ip)->jmp.a.n_bits); \ + \ + uint64_t b = (ip)->jmp.b_val; \ + \ + (thread)->ip = (a operator b) ? (ip)->jmp.ip : ((thread)->ip + 1); \ +} + +#else + +#define JMP_CMP_HI JMP_CMP_I + +#endif + #define METADATA_READ(thread, offset, n_bits) \ ({ \ uint64_t *m64_ptr = (uint64_t *)&(thread)->metadata[offset]; \ @@ -2048,6 +2257,42 @@ metadata_free(struct rte_swx_pipeline *p) /* * Instruction. */ +static int +instruction_is_jmp(struct instruction *instr) +{ + switch (instr->type) { + case INSTR_JMP: + case INSTR_JMP_VALID: + case INSTR_JMP_INVALID: + case INSTR_JMP_HIT: + case INSTR_JMP_MISS: + case INSTR_JMP_ACTION_HIT: + case INSTR_JMP_ACTION_MISS: + case INSTR_JMP_EQ: + case INSTR_JMP_EQ_S: + case INSTR_JMP_EQ_I: + case INSTR_JMP_NEQ: + case INSTR_JMP_NEQ_S: + case INSTR_JMP_NEQ_I: + case INSTR_JMP_LT: + case INSTR_JMP_LT_MH: + case INSTR_JMP_LT_HM: + case INSTR_JMP_LT_HH: + case INSTR_JMP_LT_MI: + case INSTR_JMP_LT_HI: + case INSTR_JMP_GT: + case INSTR_JMP_GT_MH: + case INSTR_JMP_GT_HM: + case INSTR_JMP_GT_HH: + case INSTR_JMP_GT_MI: + case INSTR_JMP_GT_HI: + return 1; + + default: + return 0; + } +} + static struct field * action_field_parse(struct action *action, const char *name); @@ -2136,6 +2381,12 @@ thread_ip_reset(struct rte_swx_pipeline *p, struct thread *t) t->ip = p->instructions; } +static inline void +thread_ip_set(struct thread *t, struct instruction *ip) +{ + t->ip = ip; +} + static inline void thread_ip_action_call(struct rte_swx_pipeline *p, struct thread *t, @@ -4351,141 +4602,819 @@ instr_alu_ckadd_struct_exec(struct rte_swx_pipeline *p) thread_ip_inc(p); } -#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 +/* + * jmp. + */ +static struct action * +action_find(struct rte_swx_pipeline *p, const char *name); static int -instr_translate(struct rte_swx_pipeline *p, - struct action *action, - char *string, - struct instruction *instr, - struct instruction_data *data) +instr_jmp_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) { - char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; - int n_tokens = 0, tpos = 0; + CHECK(n_tokens == 2, EINVAL); - /* Parse the instruction string into tokens. */ - for ( ; ; ) { - char *token; + strcpy(data->jmp_label, tokens[1]); - token = strtok_r(string, " \t\v", &string); - if (!token) - break; + instr->type = INSTR_JMP; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); +static int +instr_jmp_valid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - tokens[n_tokens] = token; - n_tokens++; - } + CHECK(n_tokens == 3, EINVAL); - CHECK(n_tokens, EINVAL); + strcpy(data->jmp_label, tokens[1]); - /* Handle the optional instruction label. */ - if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { - strcpy(data->label, tokens[0]); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - tpos += 2; - CHECK(n_tokens - tpos, EINVAL); - } + instr->type = INSTR_JMP_VALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - /* Identify the instruction type. */ - if (!strcmp(tokens[tpos], "rx")) - return instr_rx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_invalid_translate(struct rte_swx_pipeline *p, + struct action *action __rte_unused, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct header *h; - if (!strcmp(tokens[tpos], "tx")) - return instr_tx_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "extract")) - return instr_hdr_extract_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "emit")) - return instr_hdr_emit_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + h = header_parse(p, tokens[2]); + CHECK(h, EINVAL); - if (!strcmp(tokens[tpos], "validate")) - return instr_hdr_validate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_INVALID; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.header_id = h->id; + return 0; +} - if (!strcmp(tokens[tpos], "invalidate")) - return instr_hdr_invalidate_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_hit_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "mov")) - return instr_mov_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "dma")) - return instr_dma_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "add")) - return instr_alu_add_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_miss_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + CHECK(!action, EINVAL); + CHECK(n_tokens == 2, EINVAL); - if (!strcmp(tokens[tpos], "sub")) - return instr_alu_sub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + strcpy(data->jmp_label, tokens[1]); - if (!strcmp(tokens[tpos], "ckadd")) - return instr_alu_ckadd_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); + instr->type = INSTR_JMP_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + return 0; +} - if (!strcmp(tokens[tpos], "cksub")) - return instr_alu_cksub_translate(p, - action, - &tokens[tpos], - n_tokens - tpos, - instr, - data); +static int +instr_jmp_action_hit_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; - if (!strcmp(tokens[tpos], "and")) - return instr_alu_and_translate(p, + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_HIT; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_action_miss_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + struct action *a; + + CHECK(!action, EINVAL); + CHECK(n_tokens == 3, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + a = action_find(p, tokens[2]); + CHECK(a, EINVAL); + + instr->type = INSTR_JMP_ACTION_MISS; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.action_id = a->id; + return 0; +} + +static int +instr_jmp_eq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_EQ or JMP_EQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_EQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_EQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_EQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_EQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_neq_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_NEQ or JMP_NEQ_S. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_NEQ; + if ((a[0] == 'h' && b[0] != 'h') || + (a[0] != 'h' && b[0] == 'h')) + instr->type = INSTR_JMP_NEQ_S; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_NEQ_I. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + if (a[0] == 'h') + b_val = htonl(b_val); + + instr->type = INSTR_JMP_NEQ_I; + instr->jmp.ip = NULL; /* Resolved later. */ + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_lt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_LT, JMP_LT_MH, JMP_LT_HM, JMP_LT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_LT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_LT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_LT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_LT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_LT_MI, JMP_LT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_LT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_LT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static int +instr_jmp_gt_translate(struct rte_swx_pipeline *p, + struct action *action, + char **tokens, + int n_tokens, + struct instruction *instr, + struct instruction_data *data) +{ + char *a = tokens[2], *b = tokens[3]; + struct field *fa, *fb; + uint32_t a_struct_id, b_struct_id, b_val; + + CHECK(n_tokens == 4, EINVAL); + + strcpy(data->jmp_label, tokens[1]); + + fa = struct_field_parse(p, action, a, &a_struct_id); + CHECK(fa, EINVAL); + + /* JMP_GT, JMP_GT_MH, JMP_GT_HM, JMP_GT_HH. */ + fb = struct_field_parse(p, action, b, &b_struct_id); + if (fb) { + instr->type = INSTR_JMP_GT; + if (a[0] == 'h' && b[0] == 'm') + instr->type = INSTR_JMP_GT_HM; + if (a[0] == 'm' && b[0] == 'h') + instr->type = INSTR_JMP_GT_MH; + if (a[0] == 'h' && b[0] == 'h') + instr->type = INSTR_JMP_GT_HH; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b.struct_id = (uint8_t)b_struct_id; + instr->jmp.b.n_bits = fb->n_bits; + instr->jmp.b.offset = fb->offset / 8; + return 0; + } + + /* JMP_GT_MI, JMP_GT_HI. */ + b_val = strtoul(b, &b, 0); + CHECK(!b[0], EINVAL); + + instr->type = INSTR_JMP_GT_MI; + if (a[0] == 'h') + instr->type = INSTR_JMP_GT_HI; + instr->jmp.ip = NULL; /* Resolved later. */ + + instr->jmp.a.struct_id = (uint8_t)a_struct_id; + instr->jmp.a.n_bits = fa->n_bits; + instr->jmp.a.offset = fa->offset / 8; + instr->jmp.b_val = (uint32_t)b_val; + return 0; +} + +static inline void +instr_jmp_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmp\n", p->thread_id); + + thread_ip_set(t, ip->jmp.ip); +} + +static inline void +instr_jmp_valid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_invalid_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + uint32_t header_id = ip->jmp.header_id; + + TRACE("[Thread %2u] jmpnv\n", p->thread_id); + + t->ip = HEADER_VALID(t, header_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {t->ip + 1, ip->jmp.ip}; + + TRACE("[Thread %2u] jmph\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + struct instruction *ip_next[] = {ip->jmp.ip, t->ip + 1}; + + TRACE("[Thread %2u] jmpnh\n", p->thread_id); + + t->ip = ip_next[t->hit]; +} + +static inline void +instr_jmp_action_hit_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpa\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? ip->jmp.ip : (t->ip + 1); +} + +static inline void +instr_jmp_action_miss_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpna\n", p->thread_id); + + t->ip = (ip->jmp.action_id == t->action_id) ? (t->ip + 1) : ip->jmp.ip; +} + +static inline void +instr_jmp_eq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq\n", p->thread_id); + + JMP_CMP(t, ip, ==); +} + +static inline void +instr_jmp_eq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, ==); +} + +static inline void +instr_jmp_eq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpeq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, ==); +} + +static inline void +instr_jmp_neq_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq\n", p->thread_id); + + JMP_CMP(t, ip, !=); +} + +static inline void +instr_jmp_neq_s_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (s)\n", p->thread_id); + + JMP_CMP_S(t, ip, !=); +} + +static inline void +instr_jmp_neq_i_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpneq (i)\n", p->thread_id); + + JMP_CMP_I(t, ip, !=); +} + +static inline void +instr_jmp_lt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt\n", p->thread_id); + + JMP_CMP(t, ip, <); +} + +static inline void +instr_jmp_lt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, <); +} + +static inline void +instr_jmp_lt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, <); +} + +static inline void +instr_jmp_lt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, <); +} + +static inline void +instr_jmp_lt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, <); +} + +static inline void +instr_jmp_lt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmplt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, <); +} + +static inline void +instr_jmp_gt_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt\n", p->thread_id); + + JMP_CMP(t, ip, >); +} + +static inline void +instr_jmp_gt_mh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mh)\n", p->thread_id); + + JMP_CMP_MH(t, ip, >); +} + +static inline void +instr_jmp_gt_hm_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hm)\n", p->thread_id); + + JMP_CMP_HM(t, ip, >); +} + +static inline void +instr_jmp_gt_hh_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hh)\n", p->thread_id); + + JMP_CMP_HH(t, ip, >); +} + +static inline void +instr_jmp_gt_mi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (mi)\n", p->thread_id); + + JMP_CMP_MI(t, ip, >); +} + +static inline void +instr_jmp_gt_hi_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + TRACE("[Thread %2u] jmpgt (hi)\n", p->thread_id); + + JMP_CMP_HI(t, ip, >); +} + +/* + * return. + */ +static int +instr_return_translate(struct rte_swx_pipeline *p __rte_unused, + struct action *action, + char **tokens __rte_unused, + int n_tokens, + struct instruction *instr, + struct instruction_data *data __rte_unused) +{ + CHECK(action, EINVAL); + CHECK(n_tokens == 1, EINVAL); + + instr->type = INSTR_RETURN; + return 0; +} + +static inline void +instr_return_exec(struct rte_swx_pipeline *p) +{ + struct thread *t = &p->threads[p->thread_id]; + + TRACE("[Thread %2u] return\n", p->thread_id); + + t->ip = t->ret; +} + +#define RTE_SWX_INSTRUCTION_TOKENS_MAX 16 + +static int +instr_translate(struct rte_swx_pipeline *p, + struct action *action, + char *string, + struct instruction *instr, + struct instruction_data *data) +{ + char *tokens[RTE_SWX_INSTRUCTION_TOKENS_MAX]; + int n_tokens = 0, tpos = 0; + + /* Parse the instruction string into tokens. */ + for ( ; ; ) { + char *token; + + token = strtok_r(string, " \t\v", &string); + if (!token) + break; + + CHECK(n_tokens < RTE_SWX_INSTRUCTION_TOKENS_MAX, EINVAL); + + tokens[n_tokens] = token; + n_tokens++; + } + + CHECK(n_tokens, EINVAL); + + /* Handle the optional instruction label. */ + if ((n_tokens >= 2) && !strcmp(tokens[1], ":")) { + strcpy(data->label, tokens[0]); + + tpos += 2; + CHECK(n_tokens - tpos, EINVAL); + } + + /* Identify the instruction type. */ + if (!strcmp(tokens[tpos], "rx")) + return instr_rx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "tx")) + return instr_tx_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "extract")) + return instr_hdr_extract_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "emit")) + return instr_hdr_emit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "validate")) + return instr_hdr_validate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "invalidate")) + return instr_hdr_invalidate_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "mov")) + return instr_mov_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "dma")) + return instr_dma_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "add")) + return instr_alu_add_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "sub")) + return instr_alu_sub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "ckadd")) + return instr_alu_ckadd_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "cksub")) + return instr_alu_cksub_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "and")) + return instr_alu_and_translate(p, action, &tokens[tpos], n_tokens - tpos, @@ -4540,9 +5469,117 @@ instr_translate(struct rte_swx_pipeline *p, instr, data); + if (!strcmp(tokens[tpos], "jmp")) + return instr_jmp_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpv")) + return instr_jmp_valid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnv")) + return instr_jmp_invalid_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmph")) + return instr_jmp_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpnh")) + return instr_jmp_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpa")) + return instr_jmp_action_hit_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpna")) + return instr_jmp_action_miss_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpeq")) + return instr_jmp_eq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpneq")) + return instr_jmp_neq_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmplt")) + return instr_jmp_lt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "jmpgt")) + return instr_jmp_gt_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + + if (!strcmp(tokens[tpos], "return")) + return instr_return_translate(p, + action, + &tokens[tpos], + n_tokens - tpos, + instr, + data); + CHECK(0, EINVAL); } +static struct instruction_data * +label_find(struct instruction_data *data, uint32_t n, const char *label) +{ + uint32_t i; + + for (i = 0; i < n; i++) + if (!strcmp(label, data[i].label)) + return &data[i]; + + return NULL; +} + static uint32_t label_is_used(struct instruction_data *data, uint32_t n, const char *label) { @@ -4590,6 +5627,32 @@ instr_label_check(struct instruction_data *instruction_data, return 0; } +static int +instr_jmp_resolve(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i; + + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + struct instruction_data *found; + + if (!instruction_is_jmp(instr)) + continue; + + found = label_find(instruction_data, + n_instructions, + data->jmp_label); + CHECK(found, EINVAL); + + instr->jmp.ip = &instr[found - instruction_data]; + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -4638,6 +5701,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_jmp_resolve(instr, data, n_instructions); + if (err) + goto error; + free(data); if (a) { @@ -4746,6 +5813,38 @@ static instr_exec_t instruction_table[] = { [INSTR_TABLE] = instr_table_exec, [INSTR_EXTERN_OBJ] = instr_extern_obj_exec, [INSTR_EXTERN_FUNC] = instr_extern_func_exec, + + [INSTR_JMP] = instr_jmp_exec, + [INSTR_JMP_VALID] = instr_jmp_valid_exec, + [INSTR_JMP_INVALID] = instr_jmp_invalid_exec, + [INSTR_JMP_HIT] = instr_jmp_hit_exec, + [INSTR_JMP_MISS] = instr_jmp_miss_exec, + [INSTR_JMP_ACTION_HIT] = instr_jmp_action_hit_exec, + [INSTR_JMP_ACTION_MISS] = instr_jmp_action_miss_exec, + + [INSTR_JMP_EQ] = instr_jmp_eq_exec, + [INSTR_JMP_EQ_S] = instr_jmp_eq_s_exec, + [INSTR_JMP_EQ_I] = instr_jmp_eq_i_exec, + + [INSTR_JMP_NEQ] = instr_jmp_neq_exec, + [INSTR_JMP_NEQ_S] = instr_jmp_neq_s_exec, + [INSTR_JMP_NEQ_I] = instr_jmp_neq_i_exec, + + [INSTR_JMP_LT] = instr_jmp_lt_exec, + [INSTR_JMP_LT_MH] = instr_jmp_lt_mh_exec, + [INSTR_JMP_LT_HM] = instr_jmp_lt_hm_exec, + [INSTR_JMP_LT_HH] = instr_jmp_lt_hh_exec, + [INSTR_JMP_LT_MI] = instr_jmp_lt_mi_exec, + [INSTR_JMP_LT_HI] = instr_jmp_lt_hi_exec, + + [INSTR_JMP_GT] = instr_jmp_gt_exec, + [INSTR_JMP_GT_MH] = instr_jmp_gt_mh_exec, + [INSTR_JMP_GT_HM] = instr_jmp_gt_hm_exec, + [INSTR_JMP_GT_HH] = instr_jmp_gt_hh_exec, + [INSTR_JMP_GT_MI] = instr_jmp_gt_mi_exec, + [INSTR_JMP_GT_HI] = instr_jmp_gt_hi_exec, + + [INSTR_RETURN] = instr_return_exec, }; static inline void -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 26/40] pipeline: add instruction verifier 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (24 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 25/40] pipeline: introduce jmp and return instructions Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 27/40] pipeline: add instruction optimizer Cristian Dumitrescu ` (13 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Instruction verifier. Executes at instruction translation time during pipeline build, i.e. initialization instead of run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 4eb1f4228..0016e0f0a 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5653,6 +5653,53 @@ instr_jmp_resolve(struct instruction *instructions, return 0; } +static int +instr_verify(struct rte_swx_pipeline *p __rte_unused, + struct action *a, + struct instruction *instr, + struct instruction_data *data __rte_unused, + uint32_t n_instructions) +{ + if (!a) { + enum instruction_type type; + uint32_t i; + + /* Check that the first instruction is rx. */ + CHECK(instr[0].type == INSTR_RX, EINVAL); + + /* Check that there is at least one tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if (instr[i].type == INSTR_TX) + break; + } + CHECK(i < n_instructions, EINVAL); + + /* Check that the last instruction is either tx or unconditional + * jump. + */ + type = instr[n_instructions - 1].type; + CHECK((type == INSTR_TX) || (type == INSTR_JMP), EINVAL); + } + + if (a) { + enum instruction_type type; + uint32_t i; + + /* Check that there is at least one return or tx instruction. */ + for (i = 0; i < n_instructions; i++) { + type = instr[i].type; + + if ((type == INSTR_RETURN) || (type == INSTR_TX)) + break; + } + CHECK(i < n_instructions, EINVAL); + } + + return 0; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5701,6 +5748,10 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + err = instr_verify(p, a, instr, data, n_instructions); + if (err) + goto error; + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 27/40] pipeline: add instruction optimizer 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (25 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 26/40] pipeline: add instruction verifier Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 28/40] pipeline: add pipeline query API Cristian Dumitrescu ` (12 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Instruction optimizer. Detects frequent patterns and replaces them with some more powerful vector-like pipeline instructions without any user effort. Executes at instruction translation, not at run-time. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.c | 226 +++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 0016e0f0a..d5c2a9c5d 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -5700,6 +5700,230 @@ instr_verify(struct rte_swx_pipeline *p __rte_unused, return 0; } +static int +instr_pattern_extract_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EXTRACT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_extract_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static int +instr_pattern_emit_many_tx_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_HDR_EMIT) + break; + + if (i == RTE_DIM(instr->io.hdr.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (!i) + return 0; + + if (instr[i].type != INSTR_TX) + return 0; + + i++; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_emit_many_tx_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + /* Any emit instruction in addition to the first one. */ + for (i = 1; i < n_instr - 1; i++) { + instr[0].type++; + instr[0].io.hdr.header_id[i] = instr[i].io.hdr.header_id[0]; + instr[0].io.hdr.struct_id[i] = instr[i].io.hdr.struct_id[0]; + instr[0].io.hdr.n_bytes[i] = instr[i].io.hdr.n_bytes[0]; + + data[i].invalid = 1; + } + + /* The TX instruction is the last one in the pattern. */ + instr[0].type++; + instr[0].io.io.offset = instr[i].io.io.offset; + instr[0].io.io.n_bits = instr[i].io.io.n_bits; + data[i].invalid = 1; +} + +static int +instr_pattern_dma_many_detect(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr, + uint32_t *n_pattern_instr) +{ + uint32_t i; + + for (i = 0; i < n_instr; i++) { + if (data[i].invalid) + break; + + if (instr[i].type != INSTR_DMA_HT) + break; + + if (i == RTE_DIM(instr->dma.dst.header_id)) + break; + + if (i && data[i].n_users) + break; + } + + if (i < 2) + return 0; + + *n_pattern_instr = i; + return 1; +} + +static void +instr_pattern_dma_many_optimize(struct instruction *instr, + struct instruction_data *data, + uint32_t n_instr) +{ + uint32_t i; + + for (i = 1; i < n_instr; i++) { + instr[0].type++; + instr[0].dma.dst.header_id[i] = instr[i].dma.dst.header_id[0]; + instr[0].dma.dst.struct_id[i] = instr[i].dma.dst.struct_id[0]; + instr[0].dma.src.offset[i] = instr[i].dma.src.offset[0]; + instr[0].dma.n_bytes[i] = instr[i].dma.n_bytes[0]; + + data[i].invalid = 1; + } +} + +static uint32_t +instr_optimize(struct instruction *instructions, + struct instruction_data *instruction_data, + uint32_t n_instructions) +{ + uint32_t i, pos = 0; + + for (i = 0; i < n_instructions; ) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + uint32_t n_instr = 0; + int detected; + + /* Extract many. */ + detected = instr_pattern_extract_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_extract_many_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* Emit many + TX. */ + detected = instr_pattern_emit_many_tx_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_emit_many_tx_optimize(instr, + data, + n_instr); + i += n_instr; + continue; + } + + /* DMA many. */ + detected = instr_pattern_dma_many_detect(instr, + data, + n_instructions - i, + &n_instr); + if (detected) { + instr_pattern_dma_many_optimize(instr, data, n_instr); + i += n_instr; + continue; + } + + /* No pattern starting at the current instruction. */ + i++; + } + + /* Eliminate the invalid instructions that have been optimized out. */ + for (i = 0; i < n_instructions; i++) { + struct instruction *instr = &instructions[i]; + struct instruction_data *data = &instruction_data[i]; + + if (data->invalid) + continue; + + if (i != pos) { + memcpy(&instructions[pos], instr, sizeof(*instr)); + memcpy(&instruction_data[pos], data, sizeof(*data)); + } + + pos++; + } + + return pos; +} + static int instruction_config(struct rte_swx_pipeline *p, struct action *a, @@ -5752,6 +5976,8 @@ instruction_config(struct rte_swx_pipeline *p, if (err) goto error; + n_instructions = instr_optimize(instr, data, n_instructions); + err = instr_jmp_resolve(instr, data, n_instructions); if (err) goto error; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 28/40] pipeline: add pipeline query API 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (26 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 27/40] pipeline: add instruction optimizer Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 29/40] pipeline: add pipeline flush Cristian Dumitrescu ` (11 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Query API to be used by the control plane to detect the configuration and state of the pipeline and its internal objects. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 10 + lib/librte_pipeline/rte_swx_ctl.h | 313 +++++++++++++++++++ lib/librte_pipeline/rte_swx_pipeline.c | 219 +++++++++++++ 3 files changed, 542 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 793957eb9..bb992fdd0 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -76,4 +76,14 @@ EXPERIMENTAL { rte_swx_pipeline_run; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_info_get; + rte_swx_ctl_pipeline_numa_node_get; + rte_swx_ctl_pipeline_port_in_stats_read; + rte_swx_ctl_pipeline_port_out_stats_read; + rte_swx_ctl_action_info_get; + rte_swx_ctl_action_arg_info_get; + rte_swx_ctl_table_info_get; + rte_swx_ctl_table_match_field_info_get; + rte_swx_ctl_table_action_info_get; + rte_swx_ctl_table_ops_get; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index c824ab56f..bdcc24cee 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -18,8 +18,321 @@ extern "C" { #include <rte_compat.h> +#include "rte_swx_port.h" #include "rte_swx_table.h" +struct rte_swx_pipeline; + +/** Name size. */ +#ifndef RTE_SWX_CTL_NAME_SIZE +#define RTE_SWX_CTL_NAME_SIZE 64 +#endif + +/* + * Pipeline Query API. + */ + +/** Pipeline info. */ +struct rte_swx_ctl_pipeline_info { + /** Number of input ports. */ + uint32_t n_ports_in; + + /** Number of input ports. */ + uint32_t n_ports_out; + + /** Number of actions. */ + uint32_t n_actions; + + /** Number of tables. */ + uint32_t n_tables; +}; + +/** + * Pipeline info get + * + * @param[in] p + * Pipeline handle. + * @param[out] pipeline + * Pipeline info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline); + +/** + * Pipeline NUMA node get + * + * @param[in] p + * Pipeline handle. + * @param[out] numa_node + * Pipeline NUMA node. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, + int *numa_node); + +/* + * Ports Query API. + */ + +/** + * Input port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_in* - 1). + * @param[out] stats + * Input port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats); + +/** + * Output port statistics counters read + * + * @param[in] p + * Pipeline handle. + * @param[in] port_id + * Port ID (0 .. *n_ports_out* - 1). + * @param[out] stats + * Output port stats. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats); + +/* + * Action Query API. + */ + +/** Action info. */ +struct rte_swx_ctl_action_info { + /** Action name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of action arguments. */ + uint32_t n_args; +}; + +/** + * Action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[out] action + * Action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action); + +/** Action argument info. */ +struct rte_swx_ctl_action_arg_info { + /** Action argument name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Action argument size (in bits). */ + uint32_t n_bits; +}; + +/** + * Action argument info get + * + * @param[in] p + * Pipeline handle. + * @param[in] action_id + * Action ID (0 .. *n_actions* - 1). + * @param[in] action_arg_id + * Action ID (0 .. *n_args* - 1). + * @param[out] action + * Action argument info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg); + +/* + * Table Query API. + */ + +/** Table info. */ +struct rte_swx_ctl_table_info { + /** Table name. */ + char name[RTE_SWX_CTL_NAME_SIZE]; + + /** Table creation arguments. */ + char args[RTE_SWX_CTL_NAME_SIZE]; + + /** Number of match fields. */ + uint32_t n_match_fields; + + /** Number of actions. */ + uint32_t n_actions; + + /** Non-zero (true) when the default action is constant, therefore it + * cannot be changed; zero (false) when the default action not constant, + * therefore it can be changed. + */ + int default_action_is_const; + + /** Table size parameter. */ + uint32_t size; +}; + +/** + * Table info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables* - 1). + * @param[out] table + * Table info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table); + +/** Table match field info. + * + * If (n_bits, offset) are known for all the match fields of the table, then the + * table (key_offset, key_size, key_mask0) can be computed. + */ +struct rte_swx_ctl_table_match_field_info { + /** Match type of the current match field. */ + enum rte_swx_table_match_type match_type; + + /** Non-zero (true) when the current match field is part of a registered + * header, zero (false) when it is part of the registered meta-data. + */ + int is_header; + + /** Match field size (in bits). */ + uint32_t n_bits; + + /** Match field offset within its parent struct (one of the headers or + * the meta-data). + */ + uint32_t offset; +}; + +/** + * Table match field info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] match_field_id + * Match field ID (0 .. *n_match_fields* - 1). + * @param[out] match_field + * Table match field info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field); + +/** Table action info. */ +struct rte_swx_ctl_table_action_info { + /** Action ID. */ + uint32_t action_id; +}; + +/** + * Table action info get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[in] table_action_id + * Action index within the set of table actions (0 .. table n_actions - 1). + * Not to be confused with the action ID, which works at the pipeline level + * (0 .. pipeline n_actions - 1), which is precisely what this function + * returns as part of *table_action*. + * @param[out] table_action + * Table action info. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action); + +/** + * Table operations get + * + * @param[in] p + * Pipeline handle. + * @param[in] table_id + * Table ID (0 .. *n_tables*). + * @param[out] table_ops + * Table operations. Only valid when function returns success and *is_stub* is + * zero (false). + * @param[out] is_stub + * A stub table is a table with no match fields. No "regular" table entries + * (i.e. entries other than the default entry) can be added to such a table, + * therefore the lookup opeation always results in lookup miss. Non-zero + * (true) when the current table is a stub table, zero (false) otherwise. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub); + /* * Table Update API. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index d5c2a9c5d..6963f63a2 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6152,6 +6152,18 @@ action_find(struct rte_swx_pipeline *p, const char *name) return NULL; } +static struct action * +action_find_by_id(struct rte_swx_pipeline *p, uint32_t id) +{ + struct action *action = NULL; + + TAILQ_FOREACH(action, &p->actions, node) + if (action->id == id) + return action; + + return NULL; +} + static struct field * action_field_find(struct action *a, const char *name) { @@ -6942,6 +6954,177 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) /* * Control. */ +int +rte_swx_ctl_pipeline_info_get(struct rte_swx_pipeline *p, + struct rte_swx_ctl_pipeline_info *pipeline) +{ + struct action *action; + struct table *table; + uint32_t n_actions = 0, n_tables = 0; + + if (!p || !pipeline) + return -EINVAL; + + TAILQ_FOREACH(action, &p->actions, node) + n_actions++; + + TAILQ_FOREACH(table, &p->tables, node) + n_tables++; + + pipeline->n_ports_in = p->n_ports_in; + pipeline->n_ports_out = p->n_ports_out; + pipeline->n_actions = n_actions; + pipeline->n_tables = n_tables; + + return 0; +} + +int +rte_swx_ctl_pipeline_numa_node_get(struct rte_swx_pipeline *p, int *numa_node) +{ + if (!p || !numa_node) + return -EINVAL; + + *numa_node = p->numa_node; + return 0; +} + +int +rte_swx_ctl_action_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + struct rte_swx_ctl_action_info *action) +{ + struct action *a = NULL; + + if (!p || (action_id >= p->n_actions) || !action) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a) + return -EINVAL; + + strcpy(action->name, a->name); + action->n_args = a->st ? a->st->n_fields : 0; + return 0; +} + +int +rte_swx_ctl_action_arg_info_get(struct rte_swx_pipeline *p, + uint32_t action_id, + uint32_t action_arg_id, + struct rte_swx_ctl_action_arg_info *action_arg) +{ + struct action *a = NULL; + struct field *arg = NULL; + + if (!p || (action_id >= p->n_actions) || !action_arg) + return -EINVAL; + + a = action_find_by_id(p, action_id); + if (!a || !a->st || (action_arg_id >= a->st->n_fields)) + return -EINVAL; + + arg = &a->st->fields[action_arg_id]; + strcpy(action_arg->name, arg->name); + action_arg->n_bits = arg->n_bits; + + return 0; +} + +int +rte_swx_ctl_table_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_ctl_table_info *table) +{ + struct table *t = NULL; + + if (!p || !table) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + strcpy(table->name, t->name); + strcpy(table->args, t->args); + table->n_match_fields = t->n_fields; + table->n_actions = t->n_actions; + table->default_action_is_const = t->default_action_is_const; + table->size = t->size; + return 0; +} + +int +rte_swx_ctl_table_match_field_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t match_field_id, + struct rte_swx_ctl_table_match_field_info *match_field) +{ + struct table *t; + struct match_field *f; + + if (!p || (table_id >= p->n_tables) || !match_field) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (match_field_id >= t->n_fields)) + return -EINVAL; + + f = &t->fields[match_field_id]; + match_field->match_type = f->match_type; + match_field->is_header = t->is_header; + match_field->n_bits = f->field->n_bits; + match_field->offset = f->field->offset; + + return 0; +} + +int +rte_swx_ctl_table_action_info_get(struct rte_swx_pipeline *p, + uint32_t table_id, + uint32_t table_action_id, + struct rte_swx_ctl_table_action_info *table_action) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables) || !table_action) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t || (table_action_id >= t->n_actions)) + return -EINVAL; + + table_action->action_id = t->actions[table_action_id]->id; + + return 0; +} + +int +rte_swx_ctl_table_ops_get(struct rte_swx_pipeline *p, + uint32_t table_id, + struct rte_swx_table_ops *table_ops, + int *is_stub) +{ + struct table *t; + + if (!p || (table_id >= p->n_tables)) + return -EINVAL; + + t = table_find_by_id(p, table_id); + if (!t) + return -EINVAL; + + if (t->type) { + if (table_ops) + memcpy(table_ops, &t->type->ops, sizeof(*table_ops)); + *is_stub = 0; + } else { + *is_stub = 1; + } + + return 0; +} + int rte_swx_pipeline_table_state_get(struct rte_swx_pipeline *p, struct rte_swx_table_state **table_state) @@ -6963,3 +7146,39 @@ rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, p->table_state = table_state; return 0; } + +int +rte_swx_ctl_pipeline_port_in_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_in_stats *stats) +{ + struct port_in *port; + + if (!p || !stats) + return -EINVAL; + + port = port_in_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} + +int +rte_swx_ctl_pipeline_port_out_stats_read(struct rte_swx_pipeline *p, + uint32_t port_id, + struct rte_swx_port_out_stats *stats) +{ + struct port_out *port; + + if (!p || !stats) + return -EINVAL; + + port = port_out_find(p, port_id); + if (!port) + return -EINVAL; + + port->type->ops.stats_read(port->obj, stats); + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 29/40] pipeline: add pipeline flush 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (27 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 28/40] pipeline: add pipeline query API Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 30/40] pipeline: add instruction description Cristian Dumitrescu ` (10 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Flush the packets currently buffered by the pipeline output ports. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_pipeline_version.map | 1 + lib/librte_pipeline/rte_swx_pipeline.c | 13 +++++++++++++ lib/librte_pipeline/rte_swx_pipeline.h | 12 ++++++++++++ 3 files changed, 26 insertions(+) diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index bb992fdd0..730e11a0c 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -74,6 +74,7 @@ EXPERIMENTAL { rte_swx_pipeline_build; rte_swx_pipeline_free; rte_swx_pipeline_run; + rte_swx_pipeline_flush; rte_swx_pipeline_table_state_get; rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; diff --git a/lib/librte_pipeline/rte_swx_pipeline.c b/lib/librte_pipeline/rte_swx_pipeline.c index 6963f63a2..64fc187c4 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.c +++ b/lib/librte_pipeline/rte_swx_pipeline.c @@ -6951,6 +6951,19 @@ rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions) instr_exec(p); } +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p) +{ + uint32_t i; + + for (i = 0; i < p->n_ports_out; i++) { + struct port_out_runtime *port = &p->out[i]; + + if (port->flush) + port->flush(port->obj); + } +} + /* * Control. */ diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index fb83a8820..203e394d6 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -547,6 +547,18 @@ void rte_swx_pipeline_run(struct rte_swx_pipeline *p, uint32_t n_instructions); +/** + * Pipeline flush + * + * Flush all output ports of the pipeline. + * + * @param[in] p + * Pipeline handle. + */ +__rte_experimental +void +rte_swx_pipeline_flush(struct rte_swx_pipeline *p); + /** * Pipeline free * -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 30/40] pipeline: add instruction description 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (28 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 29/40] pipeline: add pipeline flush Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 31/40] pipeline: add table update high level API Cristian Dumitrescu ` (9 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Added instruction set reference table. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/rte_swx_pipeline.h | 109 +++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/lib/librte_pipeline/rte_swx_pipeline.h b/lib/librte_pipeline/rte_swx_pipeline.h index 203e394d6..6da5710af 100644 --- a/lib/librte_pipeline/rte_swx_pipeline.h +++ b/lib/librte_pipeline/rte_swx_pipeline.h @@ -345,6 +345,115 @@ int rte_swx_pipeline_packet_metadata_register(struct rte_swx_pipeline *p, const char *struct_type_name); +/* + * Instructions + */ + +/** + * Instruction operands: + * + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| | Description | Format | DST | SRC |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| hdr | Header | h.header | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| act | Action | ACTION | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| tbl | Table | TABLE | | |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| H | Header field | h.header.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| M | Meta-data field | m.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| E | Extern obj mailbox field | e.ext_obj.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| F | Extern func mailbox field | f.ext_func.field | YES | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| T | Table action data field | t.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + *<pre>| I | Immediate value (32-bit) | h.header.field | NO | YES |</pre> + *<pre>+-----+---------------------------+------------------+-----+-----+</pre> + * + * Instruction set: + * + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| Instr. | Instruction | Instruction | 1st | 2nd |</pre> + *<pre>| Name | Description | Format | opnd.| opnd. |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| rx | Receive one pkt | rx m.port_in | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| tx | Transmit one pkt | tx m.port_out | M | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extract | Extract one hdr | extract h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| emit | Emit one hdr | emit h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| validate | Validate one hdr | validate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| invalidate | Invalidate one hdr | invalidate h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| mov | dst = src | mov dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| dma | memcpy(h.hdr, | dma h.hdr t.field | hdr | T |</pre> + *<pre>| | &t.field, | | | |</pre> + *<pre>| | sizeof(h.hdr) | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| add | dst += src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| sub | dst -= src | add dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| ckadd | Checksum add: dst = | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst '+ src[0:1] '+ | | | or hdr |</pre> + *<pre>| | src[2:3] '+ ... | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| cksub | Checksum subtract: | add dst src | HMEF | HMEFTI |</pre> + *<pre>| | dst = dst '- src | | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| and | dst &= src | and dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| or | dst |= src | or dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| xor | dst ^= src | xor dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shl | dst <<= src | shl dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| shr | dst >>= src | shr dst src | HMEF | HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| table | Table lookup | table TABLE | tbl | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| extern | Ext obj member func | extern e.obj.mfunc| ext | |</pre> + *<pre>| | call or ext func call| extern f.func | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmp | Unconditional jump | jmp LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpv | Jump if hdr is valid | jmpv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnv | Jump if hdr is inval | jmpnv LABEL h.hdr | hdr | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmph | Jump if tbl lkp hit | jmph LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpnh | Jump if tbl lkp miss | jmpnh LABEL | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpa | Jump if action run | jmpa LABEL ACTION | act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpna | Jump if act not run | jmpna LABEL ACTION| act | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpeq | Jump if (a == b) | jmpeq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpneq | Jump if (a != b) | jmpneq LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmplt | Jump if (a < b) | jmplt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| jmpgt | Jump if (a > b) | jmpgt LABEL a b | HMEFT| HMEFTI |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + *<pre>| return | Return from action | return | | |</pre> + *<pre>+------------+----------------------+-------------------+------+--------+</pre> + * + * At initialization time, the pipeline and action instructions (including the + * symbolic name operands) are translated to internal data structures that are + * used at run-time. + */ + /* * Pipeline action */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 31/40] pipeline: add table update high level API 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (29 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 30/40] pipeline: add instruction description Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 32/40] port: add ethernet device port Cristian Dumitrescu ` (8 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev High-level transaction-oriented API for pipeline table updates. It supports multi-table atomic updates, i.e. multiple tables can be updated in a single step with only the before and after table set visible to the packets. Uses the lower-level table update mechanisms. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_pipeline/Makefile | 1 + lib/librte_pipeline/meson.build | 3 +- lib/librte_pipeline/rte_pipeline_version.map | 15 +- lib/librte_pipeline/rte_swx_ctl.c | 1552 ++++++++++++++++++ lib/librte_pipeline/rte_swx_ctl.h | 170 ++ 5 files changed, 1737 insertions(+), 4 deletions(-) create mode 100644 lib/librte_pipeline/rte_swx_ctl.c diff --git a/lib/librte_pipeline/Makefile b/lib/librte_pipeline/Makefile index d214b1aeb..d4a5f77f7 100644 --- a/lib/librte_pipeline/Makefile +++ b/lib/librte_pipeline/Makefile @@ -22,6 +22,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := rte_pipeline.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_port_in_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_table_action.c SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_pipeline.c +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) += rte_swx_ctl.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PIPELINE)-include += rte_pipeline.h rte_port_in_action.h rte_table_action.h diff --git a/lib/librte_pipeline/meson.build b/lib/librte_pipeline/meson.build index d5f4d16e5..be1d9c3a4 100644 --- a/lib/librte_pipeline/meson.build +++ b/lib/librte_pipeline/meson.build @@ -4,7 +4,8 @@ sources = files('rte_pipeline.c', 'rte_port_in_action.c', 'rte_table_action.c', - 'rte_swx_pipeline.c',) + 'rte_swx_pipeline.c', + 'rte_swx_ctl.c',) headers = files('rte_pipeline.h', 'rte_port_in_action.h', 'rte_table_action.h', diff --git a/lib/librte_pipeline/rte_pipeline_version.map b/lib/librte_pipeline/rte_pipeline_version.map index 730e11a0c..ec38f0eef 100644 --- a/lib/librte_pipeline/rte_pipeline_version.map +++ b/lib/librte_pipeline/rte_pipeline_version.map @@ -1,4 +1,4 @@ -DPDK_21 { +DPDK_20.0 { global: rte_pipeline_ah_packet_drop; @@ -75,8 +75,6 @@ EXPERIMENTAL { rte_swx_pipeline_free; rte_swx_pipeline_run; rte_swx_pipeline_flush; - rte_swx_pipeline_table_state_get; - rte_swx_pipeline_table_state_set; rte_swx_ctl_pipeline_info_get; rte_swx_ctl_pipeline_numa_node_get; rte_swx_ctl_pipeline_port_in_stats_read; @@ -87,4 +85,15 @@ EXPERIMENTAL { rte_swx_ctl_table_match_field_info_get; rte_swx_ctl_table_action_info_get; rte_swx_ctl_table_ops_get; + rte_swx_pipeline_table_state_get; + rte_swx_pipeline_table_state_set; + rte_swx_ctl_pipeline_create; + rte_swx_ctl_pipeline_free; + rte_swx_ctl_pipeline_table_entry_add; + rte_swx_ctl_pipeline_table_default_entry_add; + rte_swx_ctl_pipeline_table_entry_delete; + rte_swx_ctl_pipeline_commit; + rte_swx_ctl_pipeline_abort; + rte_swx_ctl_pipeline_table_entry_read; + rte_swx_ctl_pipeline_table_fprintf; }; diff --git a/lib/librte_pipeline/rte_swx_ctl.c b/lib/librte_pipeline/rte_swx_ctl.c new file mode 100644 index 000000000..44da678c1 --- /dev/null +++ b/lib/librte_pipeline/rte_swx_ctl.c @@ -0,0 +1,1552 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <sys/queue.h> +#include <unistd.h> + +#include <rte_common.h> +#include <rte_byteorder.h> + +#include "rte_swx_ctl.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#define ntoh64(x) rte_be_to_cpu_64(x) +#define hton64(x) rte_cpu_to_be_64(x) + +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN +#define field_ntoh(val, n_bits) (ntoh64((val) << (64 - n_bits))) +#define field_hton(val, n_bits) (hton64((val) << (64 - n_bits))) +#else +#define field_ntoh(val, n_bits) (val) +#define field_hton(val, n_bits) (val) +#endif + +struct action { + struct rte_swx_ctl_action_info info; + struct rte_swx_ctl_action_arg_info *args; + uint32_t data_size; +}; + +struct table { + struct rte_swx_ctl_table_info info; + struct rte_swx_ctl_table_match_field_info *mf; + struct rte_swx_ctl_table_action_info *actions; + struct rte_swx_table_ops ops; + struct rte_swx_table_params params; + + struct rte_swx_table_entry_list entries; + struct rte_swx_table_entry_list pending_add; + struct rte_swx_table_entry_list pending_modify0; + struct rte_swx_table_entry_list pending_modify1; + struct rte_swx_table_entry_list pending_delete; + struct rte_swx_table_entry *pending_default; + + int is_stub; + uint32_t n_add; + uint32_t n_modify; + uint32_t n_delete; +}; + +struct rte_swx_ctl_pipeline { + struct rte_swx_ctl_pipeline_info info; + struct rte_swx_pipeline *p; + struct action *actions; + struct table *tables; + struct rte_swx_table_state *ts; + struct rte_swx_table_state *ts_next; + int numa_node; +}; + +static struct action * +action_find(struct rte_swx_ctl_pipeline *ctl, const char *action_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + + if (!strcmp(action_name, a->info.name)) + return a; + } + + return NULL; +} + +static void +action_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->actions) + return; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *action = &ctl->actions[i]; + + free(action->args); + } + + free(ctl->actions); + ctl->actions = NULL; +} + +static struct table * +table_find(struct rte_swx_ctl_pipeline *ctl, const char *table_name) +{ + uint32_t i; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + if (!strcmp(table_name, table->info.name)) + return table; + } + + return NULL; +} + +static int +table_params_get(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + uint8_t *key_mask = NULL; + enum rte_swx_table_match_type match_type = RTE_SWX_TABLE_MATCH_WILDCARD; + uint32_t key_size = 0, key_offset = 0, action_data_size = 0, i; + + if (table->info.n_match_fields) { + struct rte_swx_ctl_table_match_field_info *first, *last; + uint32_t i; + + first = &table->mf[0]; + last = &table->mf[table->info.n_match_fields - 1]; + + /* match_type. */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + + f = &table->mf[i]; + if (f->match_type != RTE_SWX_TABLE_MATCH_EXACT) + break; + } + + if (i == table->info.n_match_fields) + match_type = RTE_SWX_TABLE_MATCH_EXACT; + else if ((i == table->info.n_match_fields - 1) && + (last->match_type == RTE_SWX_TABLE_MATCH_LPM)) + match_type = RTE_SWX_TABLE_MATCH_LPM; + + /* key_offset. */ + key_offset = first->offset / 8; + + /* key_size. */ + key_size = (last->offset + last->n_bits - first->offset) / 8; + + /* key_mask. */ + key_mask = calloc(1, key_size); + CHECK(key_mask, ENOMEM); + + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *f; + uint32_t start; + size_t size; + + f = &table->mf[i]; + start = (f->offset - first->offset) / 8; + size = f->n_bits / 8; + + memset(&key_mask[start], 0xFF, size); + } + } + + /* action_data_size. */ + for (i = 0; i < table->info.n_actions; i++) { + uint32_t action_id = table->actions[i].action_id; + struct action *a = &ctl->actions[action_id]; + + if (a->data_size > action_data_size) + action_data_size = a->data_size; + } + + /* Fill in. */ + table->params.match_type = match_type; + table->params.key_size = key_size; + table->params.key_offset = key_offset; + table->params.key_mask0 = key_mask; + table->params.action_data_size = action_data_size; + table->params.n_keys_max = table->info.size; + + return 0; +} + +static void +table_entry_free(struct rte_swx_table_entry *entry) +{ + if (!entry) + return; + + free(entry->key); + free(entry->key_mask); + free(entry->action_data); + free(entry); +} + +static struct rte_swx_table_entry * +table_entry_alloc(struct table *table) +{ + struct rte_swx_table_entry *entry; + + entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!entry) + goto error; + + /* key, key_mask. */ + if (!table->is_stub) { + entry->key = calloc(1, table->params.key_size); + if (!entry->key) + goto error; + + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + entry->key_mask = calloc(1, table->params.key_size); + if (!entry->key_mask) + goto error; + } + } + + /* action_data. */ + if (table->params.action_data_size) { + entry->action_data = calloc(1, table->params.action_data_size); + if (!entry->action_data) + goto error; + } + + return entry; + +error: + table_entry_free(entry); + return NULL; +} + +static int +table_entry_check(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_check, + int data_check) +{ + struct table *table = &ctl->tables[table_id]; + + CHECK(entry, EINVAL); + + if (key_check) { + if (table->is_stub) { + /* key. */ + CHECK(!entry->key, EINVAL); + + /* key_mask. */ + CHECK(!entry->key_mask, EINVAL); + } else { + /* key. */ + CHECK(entry->key, EINVAL); + + /* key_mask. */ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_WILDCARD: + break; + + case RTE_SWX_TABLE_MATCH_LPM: + /* TBD Check that key mask is prefix. */ + break; + + case RTE_SWX_TABLE_MATCH_EXACT: + CHECK(!entry->key_mask, EINVAL); + break; + + default: + CHECK(0, EINVAL); + } + } + } + + if (data_check) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + CHECK(i < table->info.n_actions, EINVAL); + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + CHECK((a->data_size && entry->action_data) || + (!a->data_size && !entry->action_data), EINVAL); + } + + return 0; +} + +static struct rte_swx_table_entry * +table_entry_duplicate(struct rte_swx_ctl_pipeline *ctl, + uint32_t table_id, + struct rte_swx_table_entry *entry, + int key_duplicate, + int data_duplicate) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_entry *new_entry = NULL; + + if (!entry) + goto error; + + new_entry = calloc(1, sizeof(struct rte_swx_table_entry)); + if (!new_entry) + goto error; + + if (key_duplicate && !table->is_stub) { + /* key. */ + if (!entry->key) + goto error; + + new_entry->key = malloc(table->params.key_size); + if (!new_entry->key) + goto error; + + memcpy(new_entry->key, entry->key, table->params.key_size); + + /* key_signature. */ + new_entry->key_signature = entry->key_signature; + + /* key_mask. */ + if (table->params.match_type != RTE_SWX_TABLE_MATCH_EXACT) { + if (!entry->key_mask) + goto error; + + new_entry->key_mask = malloc(table->params.key_size); + if (!new_entry->key_mask) + goto error; + + memcpy(new_entry->key_mask, + entry->key_mask, + table->params.key_size); + } + } + + if (data_duplicate) { + struct action *a; + uint32_t i; + + /* action_id. */ + for (i = 0; i < table->info.n_actions; i++) + if (entry->action_id == table->actions[i].action_id) + break; + + if (i >= table->info.n_actions) + goto error; + + new_entry->action_id = entry->action_id; + + /* action_data. */ + a = &ctl->actions[entry->action_id]; + if (a->data_size) { + if (!entry->action_data) + goto error; + + new_entry->action_data = malloc(a->data_size); + if (!new_entry->action_data) + goto error; + + memcpy(new_entry->action_data, + entry->action_data, + a->data_size); + } + } + + return entry; + +error: + table_entry_free(new_entry); + return NULL; +} + +static int +entry_keycmp_em(struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1, + uint32_t key_size) +{ + if (e0->key_signature != e1->key_signature) + return 1; /* Not equal. */ + + if (memcmp(e0->key, e1->key, key_size)) + return 1; /* Not equal. */ + + return 0; /* Equal */ +} + +static int +entry_keycmp_wm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +entry_keycmp_lpm(struct rte_swx_table_entry *e0 __rte_unused, + struct rte_swx_table_entry *e1 __rte_unused, + uint32_t key_size __rte_unused) +{ + /* TBD */ + + return 1; /* Not equal */ +} + +static int +table_entry_keycmp(struct table *table, + struct rte_swx_table_entry *e0, + struct rte_swx_table_entry *e1) +{ + switch (table->params.match_type) { + case RTE_SWX_TABLE_MATCH_EXACT: + return entry_keycmp_em(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_WILDCARD: + return entry_keycmp_wm(e0, e1, table->params.key_size); + + case RTE_SWX_TABLE_MATCH_LPM: + return entry_keycmp_lpm(e0, e1, table->params.key_size); + + default: + return 1; /* Not equal. */ + } +} + +static struct rte_swx_table_entry * +table_entries_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->entries, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_entries_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->entries); + if (!entry) + break; + + TAILQ_REMOVE(&table->entries, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_add_find(struct table *table, struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_add, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_add_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_add, node); +} + +static void +table_pending_add_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_add); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_add, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify0_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify0, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify0_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify0, node); +} + +static void +table_pending_modify0_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify0); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify0, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_modify1_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_modify1, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_modify1_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_modify1, node); +} + +static void +table_pending_modify1_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_modify1); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_modify1, entry, node); + table_entry_free(entry); + } +} + +static struct rte_swx_table_entry * +table_pending_delete_find(struct table *table, + struct rte_swx_table_entry *entry) +{ + struct rte_swx_table_entry *e; + + TAILQ_FOREACH(e, &table->pending_delete, node) + if (!table_entry_keycmp(table, entry, e)) + return e; /* Found. */ + + return NULL; /* Not found. */ +} + +static void +table_pending_delete_admit(struct table *table) +{ + TAILQ_CONCAT(&table->entries, &table->pending_delete, node); +} + +static void +table_pending_delete_free(struct table *table) +{ + for ( ; ; ) { + struct rte_swx_table_entry *entry; + + entry = TAILQ_FIRST(&table->pending_delete); + if (!entry) + break; + + TAILQ_REMOVE(&table->pending_delete, entry, node); + table_entry_free(entry); + } +} + +static void +table_pending_default_free(struct table *table) +{ + if (!table->pending_default) + return; + + free(table->pending_default->action_data); + free(table->pending_default); + table->pending_default = NULL; +} + +static void +table_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->tables) + return; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + + free(table->mf); + free(table->actions); + free(table->params.key_mask0); + + table_entries_free(table); + table_pending_add_free(table); + table_pending_modify0_free(table); + table_pending_modify1_free(table); + table_pending_delete_free(table); + table_pending_default_free(table); + } + + free(ctl->tables); + ctl->tables = NULL; +} + +static void +table_state_free(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl->ts_next) + return; + + /* For each table, free its table state. */ + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts_next[i]; + + /* Default action data. */ + free(ts->default_action_data); + + /* Table object. */ + if (!table->is_stub && table->ops.free && ts->obj) + table->ops.free(ts->obj); + } + + free(ctl->ts_next); + ctl->ts_next = NULL; +} + +static int +table_state_create(struct rte_swx_ctl_pipeline *ctl) +{ + int status = 0; + uint32_t i; + + ctl->ts_next = calloc(ctl->info.n_tables, + sizeof(struct rte_swx_table_state)); + if (!ctl->ts_next) { + status = -ENOMEM; + goto error; + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *table = &ctl->tables[i]; + struct rte_swx_table_state *ts = &ctl->ts[i]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[i]; + + /* Table object. */ + if (!table->is_stub) { + ts_next->obj = table->ops.create(&table->params, + &table->entries, + table->info.args, + ctl->numa_node); + if (!ts_next->obj) { + status = -ENODEV; + goto error; + } + } + + /* Default action data: duplicate from current table state. */ + ts_next->default_action_data = + malloc(table->params.action_data_size); + if (!ts_next->default_action_data) { + status = -ENOMEM; + goto error; + } + + memcpy(ts_next->default_action_data, + ts->default_action_data, + table->params.action_data_size); + + ts_next->default_action_id = ts->default_action_id; + } + + return 0; + +error: + table_state_free(ctl); + return status; +} + +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl) +{ + if (!ctl) + return; + + action_free(ctl); + + table_state_free(ctl); + + table_free(ctl); + + free(ctl); +} + +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p) +{ + struct rte_swx_ctl_pipeline *ctl = NULL; + uint32_t i; + int status; + + if (!p) + goto error; + + ctl = calloc(1, sizeof(struct rte_swx_ctl_pipeline)); + if (!ctl) + goto error; + + /* info. */ + status = rte_swx_ctl_pipeline_info_get(p, &ctl->info); + if (status) + goto error; + + /* numa_node. */ + status = rte_swx_ctl_pipeline_numa_node_get(p, &ctl->numa_node); + if (status) + goto error; + + /* p. */ + ctl->p = p; + + /* actions. */ + ctl->actions = calloc(ctl->info.n_actions, sizeof(struct action)); + if (!ctl->actions) + goto error; + + for (i = 0; i < ctl->info.n_actions; i++) { + struct action *a = &ctl->actions[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_action_info_get(p, i, &a->info); + if (status) + goto error; + + /* args. */ + a->args = calloc(a->info.n_args, + sizeof(struct rte_swx_ctl_action_arg_info)); + if (!a->args) + goto error; + + for (j = 0; j < a->info.n_args; j++) { + status = rte_swx_ctl_action_arg_info_get(p, + i, + j, + &a->args[j]); + if (status) + goto error; + } + + /* data_size. */ + for (j = 0; j < a->info.n_args; j++) { + struct rte_swx_ctl_action_arg_info *info = &a->args[j]; + + a->data_size += info->n_bits; + } + + a->data_size = (a->data_size + 7) / 8; + } + + /* tables. */ + ctl->tables = calloc(ctl->info.n_tables, sizeof(struct table)); + if (!ctl->tables) + goto error; + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + + TAILQ_INIT(&t->entries); + TAILQ_INIT(&t->pending_add); + TAILQ_INIT(&t->pending_modify0); + TAILQ_INIT(&t->pending_modify1); + TAILQ_INIT(&t->pending_delete); + } + + for (i = 0; i < ctl->info.n_tables; i++) { + struct table *t = &ctl->tables[i]; + uint32_t j; + + /* info. */ + status = rte_swx_ctl_table_info_get(p, i, &t->info); + if (status) + goto error; + + /* mf. */ + t->mf = calloc(t->info.n_match_fields, + sizeof(struct rte_swx_ctl_table_match_field_info)); + if (!t->mf) + goto error; + + for (j = 0; j < t->info.n_match_fields; j++) { + status = rte_swx_ctl_table_match_field_info_get(p, + i, + j, + &t->mf[j]); + if (status) + goto error; + } + + /* actions. */ + t->actions = calloc(t->info.n_actions, + sizeof(struct rte_swx_ctl_table_action_info)); + if (!t->actions) + goto error; + + for (j = 0; j < t->info.n_actions; j++) { + status = rte_swx_ctl_table_action_info_get(p, + i, + j, + &t->actions[j]); + if (status || + t->actions[j].action_id >= ctl->info.n_actions) + goto error; + } + + /* ops, is_stub. */ + status = rte_swx_ctl_table_ops_get(p, i, &t->ops, &t->is_stub); + if (status) + goto error; + + if ((t->is_stub && t->info.n_match_fields) || + (!t->is_stub && !t->info.n_match_fields)) + goto error; + + /* params. */ + status = table_params_get(ctl, i); + if (status) + goto error; + } + + /* ts. */ + status = rte_swx_pipeline_table_state_get(p, &ctl->ts); + if (status) + goto error; + + /* ts_next. */ + status = table_state_create(ctl); + if (status) + goto error; + + return ctl; + +error: + rte_swx_ctl_pipeline_free(ctl); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry, *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + CHECK(table_name && table_name[0], EINVAL); + + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + new_entry = table_entry_duplicate(ctl, table_id, entry, 1, 1); + CHECK(new_entry, ENOMEM); + + /* The new entry is found in the table->entries list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->entries list to the + * table->pending_modify0 list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is found in the table->pending_add list: + * - Replace the entry in the table->pending_add list with the new entry + * (and free the replaced entry). + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_add, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_modify1 list: + * - Replace the entry in the table->pending_modify1 list with the new + * entry (and free the replaced entry). + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_AFTER(&table->pending_modify1, + existing_entry, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + return 0; + } + + /* The new entry is found in the table->pending_delete list: + * - Add the new entry to the table->pending_modify1 list; + * - Move the existing entry from the table->pending_delete list to the + * table->pending_modify0 list. + */ + existing_entry = table_pending_delete_find(table, entry); + if (existing_entry) { + TAILQ_INSERT_TAIL(&table->pending_modify1, + new_entry, + node); + + TAILQ_REMOVE(&table->pending_delete, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_modify0, + existing_entry, + node); + + return 0; + } + + /* The new entry is not found in any of the above lists: + * - Add the new entry to the table->pending_add list. + */ + TAILQ_INSERT_TAIL(&table->pending_add, new_entry, node); + + return 0; +} + +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *existing_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + + CHECK(entry, EINVAL); + CHECK(!table_entry_check(ctl, table_id, entry, 1, 0), EINVAL); + + /* The entry is found in the table->entries list: + * - Move the existing entry from the table->entries list to to the + * table->pending_delete list. + */ + existing_entry = table_entries_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->entries, + existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_add list: + * - Remove the entry from the table->pending_add list and free it. + */ + existing_entry = table_pending_add_find(table, entry); + if (existing_entry) { + TAILQ_REMOVE(&table->pending_add, + existing_entry, + node); + + table_entry_free(existing_entry); + } + + /* The entry is found in the table->pending_modify1 list: + * - Free the entry in the table->pending_modify1 list; + * - Move the existing entry from the table->pending_modify0 list to the + * table->pending_delete list. + */ + existing_entry = table_pending_modify1_find(table, entry); + if (existing_entry) { + struct rte_swx_table_entry *real_existing_entry; + + TAILQ_REMOVE(&table->pending_modify1, + existing_entry, + node); + + table_entry_free(existing_entry); + + real_existing_entry = table_pending_modify0_find(table, entry); + CHECK(real_existing_entry, EINVAL); /* Coverity. */ + + TAILQ_REMOVE(&table->pending_modify0, + real_existing_entry, + node); + + TAILQ_INSERT_TAIL(&table->pending_delete, + real_existing_entry, + node); + + return 0; + } + + /* The entry is found in the table->pending_delete list: + * - Do nothing: the existing entry is already in the + * table->pending_delete list, i.e. already marked for delete, so + * simply keep it there as it is. + */ + + /* The entry is not found in any of the above lists: + * - Do nothing: no existing entry to delete. + */ + + return 0; +} + +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry) +{ + struct table *table; + struct rte_swx_table_entry *new_entry; + uint32_t table_id; + + CHECK(ctl, EINVAL); + + CHECK(table_name && table_name[0], EINVAL); + table = table_find(ctl, table_name); + CHECK(table, EINVAL); + table_id = table - ctl->tables; + CHECK(!table->info.default_action_is_const, EINVAL); + + new_entry = table_entry_duplicate(ctl, table_id, entry, 0, 1); + CHECK(new_entry, ENOMEM); + + table_pending_default_free(table); + + table->pending_default = new_entry; + return 0; +} + +static int +table_rollfwd0(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Reset counters. */ + table->n_add = 0; + table->n_modify = 0; + table->n_delete = 0; + + /* Add pending rules. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_add++; + } + + /* Modify pending rules. */ + TAILQ_FOREACH(entry, &table->pending_modify1, node) { + int status; + + status = table->ops.add(ts_next->obj, entry); + if (status) + return status; + + table->n_modify++; + } + + /* Delete pending rules. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + int status; + + status = table->ops.del(ts_next->obj, entry); + if (status) + return status; + + table->n_delete++; + } + + return 0; +} + +static void +table_rollfwd1(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct action *a; + uint8_t *action_data; + uint64_t action_id; + + /* Copy the pending default entry. */ + if (!table->pending_default) + return; + + action_id = table->pending_default->action_id; + action_data = table->pending_default->action_data; + a = &ctl->actions[action_id]; + + memcpy(ts_next->default_action_data, + action_data, + a->data_size); + + ts_next->default_action_id = action_id; +} + +static void +table_rollfwd2(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Move all the pending add entries to the table, as they are now part + * of the table. + */ + table_pending_add_admit(table); + + /* Move all the pending modify1 entries to table, are they are now part + * of the table. Free up all the pending modify0 entries, as they are no + * longer part of the table. + */ + table_pending_modify1_admit(table); + table_pending_modify0_free(table); + + /* Free up all the pending delete entries, as they are no longer part of + * the table. + */ + table_pending_delete_free(table); + + /* Free up the pending default entry, as it is now part of the table. */ + table_pending_default_free(table); +} + +static void +table_rollback(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + struct rte_swx_table_state *ts_next = &ctl->ts_next[table_id]; + struct rte_swx_table_entry *entry; + + /* Add back all the entries that were just deleted. */ + TAILQ_FOREACH(entry, &table->pending_delete, node) { + if (!table->n_delete) + break; + + table->ops.add(ts_next->obj, entry); + table->n_delete--; + } + + /* Add back the old copy for all the entries that were just + * modified. + */ + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + if (!table->n_modify) + break; + + table->ops.add(ts_next->obj, entry); + table->n_modify--; + } + + /* Delete all the entries that were just added. */ + TAILQ_FOREACH(entry, &table->pending_add, node) { + if (!table->n_add) + break; + + table->ops.del(ts_next->obj, entry); + table->n_add--; + } +} + +static void +table_abort(struct rte_swx_ctl_pipeline *ctl, uint32_t table_id) +{ + struct table *table = &ctl->tables[table_id]; + + /* Free up all the pending add entries, as none of them is part of the + * table. + */ + table_pending_add_free(table); + + /* Free up all the pending modify1 entries, as none of them made it to + * the table. Add back all the pending modify0 entries, as none of them + * was deleted from the table. + */ + table_pending_modify1_free(table); + table_pending_modify0_admit(table); + + /* Add back all the pending delete entries, as none of them was deleted + * from the table. + */ + table_pending_delete_admit(table); + + /* Free up the pending default entry, as it is no longer going to be + * added to the table. + */ + table_pending_default_free(table); +} + +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, int abort_on_fail) +{ + struct rte_swx_table_state *ts; + int status = 0; + uint32_t i; + + CHECK(ctl, EINVAL); + + /* Operate the changes on the current ts_next before it becomes the new + * ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + status = table_rollfwd0(ctl, i); + if (status) + goto rollback; + } + + for (i = 0; i < ctl->info.n_tables; i++) + table_rollfwd1(ctl, i); + + /* Swap the table state for the data plane. The current ts and ts_next + * become the new ts_next and ts, respectively. + */ + rte_swx_pipeline_table_state_set(ctl->p, ctl->ts_next); + usleep(100); + ts = ctl->ts; + ctl->ts = ctl->ts_next; + ctl->ts_next = ts; + + /* Operate the changes on the current ts_next, which is the previous ts. + */ + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollfwd0(ctl, i); + table_rollfwd1(ctl, i); + table_rollfwd2(ctl, i); + } + + return 0; + +rollback: + for (i = 0; i < ctl->info.n_tables; i++) { + table_rollback(ctl, i); + if (abort_on_fail) + table_abort(ctl, i); + } + + return status; +} + +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl) +{ + uint32_t i; + + if (!ctl) + return; + + for (i = 0; i < ctl->info.n_tables; i++) + table_abort(ctl, i); +} + +#define RTE_SWX_CTL_ENTRY_TOKENS_MAX 256 + +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string) +{ + char *tokens[RTE_SWX_CTL_ENTRY_TOKENS_MAX]; + struct table *table; + struct action *action; + struct rte_swx_table_entry *entry = NULL; + char *s0 = NULL, *s; + uint32_t n_tokens = 0, arg_offset = 0, i; + + /* Check input arguments. */ + if (!ctl) + goto error; + + if (!table_name || !table_name[0]) + goto error; + + table = table_find(ctl, table_name); + if (!table) + goto error; + + if (!string || !string[0]) + goto error; + + /* Memory allocation. */ + s0 = strdup(string); + if (!s0) + goto error; + + entry = table_entry_alloc(table); + if (!entry) + goto error; + + /* Parse the string into tokens. */ + for (s = s0; ; ) { + char *token; + + token = strtok_r(s, " \f\n\r\t\v", &s); + if (!token) + break; + + if (n_tokens >= RTE_SWX_CTL_ENTRY_TOKENS_MAX) + goto error; + + tokens[n_tokens] = token; + n_tokens++; + } + + if ((n_tokens < 3 + table->info.n_match_fields) || + strcmp(tokens[0], "match") || + strcmp(tokens[1 + table->info.n_match_fields], "action")) + goto error; + + action = action_find(ctl, tokens[2 + table->info.n_match_fields]); + if (!action) + goto error; + + if (n_tokens != 3 + table->info.n_match_fields + + action->info.n_args * 2) + goto error; + + /* + * Match. + */ + for (i = 0; i < table->info.n_match_fields; i++) { + struct rte_swx_ctl_table_match_field_info *mf = &table->mf[i]; + char *mf_val = tokens[1 + i]; + uint64_t val; + + val = strtoull(mf_val, &mf_val, 0); + if (mf_val[0]) + goto error; + + /* Endianness conversion. */ + if (mf->is_header) + val = field_hton(val, mf->n_bits); + + /* Copy key and key_mask to entry. */ + memcpy(&entry->key[(mf->offset - table->mf[0].offset) / 8], + (uint8_t *)&val, + mf->n_bits / 8); + + /* TBD Set entry->key_mask for wilcard and LPM match tables. */ + } + + /* + * Action. + */ + /* action_id. */ + entry->action_id = action - ctl->actions; + + /* action_data. */ + for (i = 0; i < action->info.n_args; i++) { + struct rte_swx_ctl_action_arg_info *arg = &action->args[i]; + char *arg_name, *arg_val; + uint64_t val; + int is_nbo = 0; + + arg_name = tokens[3 + table->info.n_match_fields + i * 2]; + arg_val = tokens[3 + table->info.n_match_fields + i * 2 + 1]; + + if (strcmp(arg_name, arg->name) || + (strlen(arg_val) < 4) || + ((arg_val[0] != 'H') && (arg_val[0] != 'N')) || + (arg_val[1] != '(') || + (arg_val[strlen(arg_val) - 1] != ')')) + goto error; + + if (arg_val[0] == 'N') + is_nbo = 1; + + arg_val[strlen(arg_val) - 1] = 0; /* Remove the ')'. */ + arg_val += 2; /* Remove the "H(" or "N(". */ + + val = strtoull(arg_val, &arg_val, 0); + if (arg_val[0]) + goto error; + + /* Endianness conversion. */ + if (is_nbo) + val = field_hton(val, arg->n_bits); + + /* Copy to entry. */ + memcpy(&entry->action_data[arg_offset], + (uint8_t *)&val, + arg->n_bits / 8); + + arg_offset += arg->n_bits / 8; + } + + return entry; + +error: + table_entry_free(entry); + free(s0); + return NULL; +} + +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name) +{ + struct table *table; + struct rte_swx_table_entry *entry; + uint32_t n_entries = 0, i; + + if (!f || !ctl || !table_name || !table_name[0]) + return -EINVAL; + + table = table_find(ctl, table_name); + if (!table) + return -EINVAL; + + /* Table. */ + fprintf(f, "# Table %s: key size %u bytes, key offset %u, key mask [", + table->info.name, + table->params.key_size, + table->params.key_offset); + + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", table->params.key_mask0[i]); + + fprintf(f, "], action data size %u bytes\n", + table->params.action_data_size); + + /* Table entries. */ + TAILQ_FOREACH(entry, &table->entries, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_modify0, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + TAILQ_FOREACH(entry, &table->pending_delete, node) { + struct action *action = &ctl->actions[entry->action_id]; + + fprintf(f, "match "); + for (i = 0; i < table->params.key_size; i++) + fprintf(f, "%02x", entry->key[i]); + + fprintf(f, " action %s ", action->info.name); + for (i = 0; i < action->data_size; i++) + fprintf(f, "%02x", entry->action_data[i]); + + fprintf(f, "\n"); + n_entries++; + } + + fprintf(f, "# Table %s currently has %u entries.\n", + table_name, + n_entries); + return 0; +} diff --git a/lib/librte_pipeline/rte_swx_ctl.h b/lib/librte_pipeline/rte_swx_ctl.h index bdcc24cee..786adedb2 100644 --- a/lib/librte_pipeline/rte_swx_ctl.h +++ b/lib/librte_pipeline/rte_swx_ctl.h @@ -391,6 +391,176 @@ int rte_swx_pipeline_table_state_set(struct rte_swx_pipeline *p, struct rte_swx_table_state *table_state); +/* + * High Level Reference Table Update API. + */ + +/** Pipeline control opaque data structure. */ +struct rte_swx_ctl_pipeline; + +/** + * Pipeline control create + * + * @param[in] p + * Pipeline handle. + * @return + * Pipeline control handle, on success, or NULL, on error. + */ +__rte_experimental +struct rte_swx_ctl_pipeline * +rte_swx_ctl_pipeline_create(struct rte_swx_pipeline *p); + +/** + * Pipeline table entry add + * + * Schedule entry for addition to table or update as part of the next commit + * operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be added to the table. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table default entry add + * + * Schedule table default entry update as part of the next commit operation. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * The new table default entry. The *key* and *key_mask* entry fields are + * ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_default_entry_add(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline table entry delete + * + * Schedule entry for deletion from table as part of the next commit operation. + * Request is silently discarded if no such entry exists. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] entry + * Entry to be deleted from the table. The *action_id* and *action_data* entry + * fields are ignored. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_entry_delete(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + struct rte_swx_table_entry *entry); + +/** + * Pipeline commit + * + * Perform all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] abort_on_fail + * When non-zero (false), all the scheduled work is discarded after a failed + * commit. Otherwise, the scheduled work is still kept pending for the next + * commit. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_commit(struct rte_swx_ctl_pipeline *ctl, + int abort_on_fail); + +/** + * Pipeline abort + * + * Discard all the scheduled table work. + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_abort(struct rte_swx_ctl_pipeline *ctl); + +/** + * Pipeline table entry read + * + * Read table entry from string. + * + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @param[in] string + * String containing the table entry. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +struct rte_swx_table_entry * +rte_swx_ctl_pipeline_table_entry_read(struct rte_swx_ctl_pipeline *ctl, + const char *table_name, + const char *string); + +/** + * Pipeline table print to file + * + * Print all the table entries to file. + * + * @param[in] f + * Output file. + * @param[in] ctl + * Pipeline control handle. + * @param[in] table_name + * Table name. + * @return + * 0 on success or the following error codes otherwise: + * -EINVAL: Invalid argument. + */ +__rte_experimental +int +rte_swx_ctl_pipeline_table_fprintf(FILE *f, + struct rte_swx_ctl_pipeline *ctl, + const char *table_name); + +/** + * Pipeline control free + * + * @param[in] ctl + * Pipeline control handle. + */ +__rte_experimental +void +rte_swx_ctl_pipeline_free(struct rte_swx_ctl_pipeline *ctl); + #ifdef __cplusplus } #endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 32/40] port: add ethernet device port 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (30 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 31/40] pipeline: add table update high level API Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 33/40] port: add source and sink ports Cristian Dumitrescu ` (7 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add the Ethernet device input/output pipeline port type. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 3 +- lib/librte_port/rte_swx_port_ethdev.c | 313 ++++++++++++++++++++++++++ lib/librte_port/rte_swx_port_ethdev.h | 54 +++++ 5 files changed, 375 insertions(+), 3 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_ethdev.c create mode 100644 lib/librte_port/rte_swx_port_ethdev.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 4221618b3..682336c01 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -38,6 +38,7 @@ endif SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -56,5 +57,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 5b5fbf6c4..3d7f309bb 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -10,7 +10,8 @@ sources = files( 'rte_port_sched.c', 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', - 'rte_port_eventdev.c') + 'rte_port_eventdev.c', + 'rte_swx_port_ethdev.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -22,7 +23,8 @@ headers = files( 'rte_port_source_sink.h', 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', - 'rte_swx_port.h',) + 'rte_swx_port.h', + 'rte_swx_port_ethdev.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index bd1fbb66b..6da5c8074 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -37,5 +37,6 @@ EXPERIMENTAL { rte_port_eventdev_reader_ops; rte_port_eventdev_writer_ops; rte_port_eventdev_writer_nodrop_ops; - + rte_swx_port_ethdev_reader_ops; + rte_swx_port_ethdev_writer_ops; }; diff --git a/lib/librte_port/rte_swx_port_ethdev.c b/lib/librte_port/rte_swx_port_ethdev.c new file mode 100644 index 000000000..18d1c0b5d --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.c @@ -0,0 +1,313 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <stdint.h> + +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_ethdev.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port ETHDEV Reader + */ +struct reader { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + int n_pkts; + int pos; +}; + +static void * +reader_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_reader_params *params = args; + struct reader *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_rx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct reader)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static int +reader_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct reader *p = port; + struct rte_mbuf *m; + + if (p->pos == p->n_pkts) { + int n_pkts; + + n_pkts = rte_eth_rx_burst(p->params.port_id, + p->params.queue_id, + p->pkts, + p->params.burst_size); + if (!n_pkts) { + p->stats.n_empty++; + return 0; + } + + TRACE("[Ethdev RX port %u queue %u] %d packets in\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + p->n_pkts = n_pkts; + p->pos = 0; + } + + m = p->pkts[p->pos++]; + pkt->handle = m; + pkt->pkt = m->buf_addr; + pkt->offset = m->data_off; + pkt->length = m->pkt_len; + + TRACE("[Ethdev RX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->pos - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, + NULL, + &((uint8_t *)m->buf_addr)[m->data_off], + m->data_len); + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + return 1; +} + +static void +reader_free(void *port) +{ + struct reader *p = port; + int i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) { + struct rte_mbuf *pkt = p->pkts[i]; + + rte_pktmbuf_free(pkt); + } + + free(p->pkts); + free(p); +} + +static void +reader_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct reader *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Port ETHDEV Writer + */ +struct writer { + struct { + uint16_t port_id; + uint16_t queue_id; + uint32_t burst_size; + } params; + struct rte_swx_port_out_stats stats; + + struct rte_mbuf **pkts; + int n_pkts; +}; + +static void * +writer_create(void *args) +{ + struct rte_eth_dev_info info; + struct rte_swx_port_ethdev_writer_params *params = args; + struct writer *p; + int status; + uint16_t port_id; + + /* Check input parameters. */ + CHECK(params); + + CHECK(params->dev_name); + status = rte_eth_dev_get_port_by_name(params->dev_name, &port_id); + CHECK(!status); + + status = rte_eth_dev_info_get(port_id, &info); + CHECK((status == -ENOTSUP) || (params->queue_id < info.nb_tx_queues)); + + CHECK(params->burst_size); + + /* Memory allocation. */ + p = calloc(1, sizeof(struct writer)); + CHECK(p); + + p->pkts = calloc(params->burst_size, sizeof(struct rte_mbuf *)); + if (!p->pkts) { + free(p); + CHECK(0); + } + + /* Initialization. */ + p->params.port_id = port_id; + p->params.queue_id = params->queue_id; + p->params.burst_size = params->burst_size; + + return p; +} + +static void +__writer_flush(struct writer *p) +{ + int n_pkts; + + for (n_pkts = 0; ; ) { + n_pkts += rte_eth_tx_burst(p->params.port_id, + p->params.queue_id, + p->pkts + n_pkts, + p->n_pkts - n_pkts); + + TRACE("[Ethdev TX port %u queue %u] %d packets out\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + n_pkts); + + if (n_pkts == p->n_pkts) + break; + } + + p->n_pkts = 0; +} + +static void +writer_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct writer *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Ethdev TX port %u queue %u] Pkt %d (%u bytes at offset %u)\n", + (uint32_t)p->params.port_id, + (uint32_t)p->params.queue_id, + p->n_pkts - 1, + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + p->pkts[p->n_pkts++] = m; + if (p->n_pkts == (int)p->params.burst_size) + __writer_flush(p); +} + +static void +writer_flush(void *port) +{ + struct writer *p = port; + + if (p->n_pkts) + __writer_flush(p); +} + +static void +writer_free(void *port) +{ + struct writer *p = port; + + if (!p) + return; + + writer_flush(p); + free(p->pkts); + free(port); +} + +static void +writer_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct writer *p = port; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops = { + .create = reader_create, + .free = reader_free, + .pkt_rx = reader_pkt_rx, + .stats_read = reader_stats_read, +}; + +struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops = { + .create = writer_create, + .free = writer_free, + .pkt_tx = writer_pkt_tx, + .flush = writer_flush, + .stats_read = writer_stats_read, +}; diff --git a/lib/librte_port/rte_swx_port_ethdev.h b/lib/librte_port/rte_swx_port_ethdev.h new file mode 100644 index 000000000..cbc2d7b21 --- /dev/null +++ b/lib/librte_port/rte_swx_port_ethdev.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ +#define __INCLUDE_RTE_SWX_PORT_ETHDEV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Ethernet Device Input and Output Ports + */ + +#include <stdint.h> + +#include "rte_swx_port.h" + +/** Ethernet device input port (reader) creation parameters. */ +struct rte_swx_port_ethdev_reader_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device receive queue ID. */ + uint16_t queue_id; + + /** Ethernet device receive burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device reader operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_ethdev_reader_ops; + +/** Ethernet device output port (writer) creation parameters. */ +struct rte_swx_port_ethdev_writer_params { + /** Name of a valid and fully configured Ethernet device. */ + const char *dev_name; + + /** Ethernet device transmit queue ID. */ + uint16_t queue_id; + + /** Ethernet device transmit burst size. */ + uint32_t burst_size; +}; + +/** Ethernet device writer operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_ethdev_writer_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 33/40] port: add source and sink ports 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (31 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 32/40] port: add ethernet device port Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 34/40] table: add exact match table Cristian Dumitrescu ` (6 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add the PCAP file-based source (input) and sink (output) pipeline port types. The sink port is typically used to implement the packet drop pipeline action. Used under the hood by the pipeline rx and tx instructions. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_port/Makefile | 2 + lib/librte_port/meson.build | 6 +- lib/librte_port/rte_port_version.map | 2 + lib/librte_port/rte_swx_port_source_sink.c | 335 +++++++++++++++++++++ lib/librte_port/rte_swx_port_source_sink.h | 57 ++++ 5 files changed, 400 insertions(+), 2 deletions(-) create mode 100644 lib/librte_port/rte_swx_port_source_sink.c create mode 100644 lib/librte_port/rte_swx_port_source_sink.h diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index 682336c01..e4a9865d9 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -39,6 +39,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_source_sink.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_sym_crypto.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_port_eventdev.c SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_ethdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PORT) += rte_swx_port_source_sink.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port.h @@ -58,5 +59,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_sym_crypto.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_eventdev.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port.h SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_ethdev.h +SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_swx_port_source_sink.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/meson.build b/lib/librte_port/meson.build index 3d7f309bb..9bbae28b7 100644 --- a/lib/librte_port/meson.build +++ b/lib/librte_port/meson.build @@ -11,7 +11,8 @@ sources = files( 'rte_port_source_sink.c', 'rte_port_sym_crypto.c', 'rte_port_eventdev.c', - 'rte_swx_port_ethdev.c',) + 'rte_swx_port_ethdev.c', + 'rte_swx_port_source_sink.c',) headers = files( 'rte_port_ethdev.h', 'rte_port_fd.h', @@ -24,7 +25,8 @@ headers = files( 'rte_port_sym_crypto.h', 'rte_port_eventdev.h', 'rte_swx_port.h', - 'rte_swx_port_ethdev.h',) + 'rte_swx_port_ethdev.h', + 'rte_swx_port_source_sink.h',) deps += ['ethdev', 'sched', 'ip_frag', 'cryptodev', 'eventdev'] if dpdk_conf.has('RTE_PORT_PCAP') diff --git a/lib/librte_port/rte_port_version.map b/lib/librte_port/rte_port_version.map index 6da5c8074..eb4dd9347 100644 --- a/lib/librte_port/rte_port_version.map +++ b/lib/librte_port/rte_port_version.map @@ -39,4 +39,6 @@ EXPERIMENTAL { rte_port_eventdev_writer_nodrop_ops; rte_swx_port_ethdev_reader_ops; rte_swx_port_ethdev_writer_ops; + rte_swx_port_source_ops; + rte_swx_port_sink_ops; }; diff --git a/lib/librte_port/rte_swx_port_source_sink.c b/lib/librte_port/rte_swx_port_source_sink.c new file mode 100644 index 000000000..dcec9025a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.c @@ -0,0 +1,335 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdint.h> +#include <stdlib.h> +#include <string.h> +#ifdef RTE_PORT_PCAP +#include <pcap.h> +#endif +#include <sys/time.h> + +#include <rte_mbuf.h> +#include <rte_hexdump.h> + +#include "rte_swx_port_source_sink.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) \ + return NULL; \ +} while (0) + +#ifndef TRACE_LEVEL +#define TRACE_LEVEL 0 +#endif + +#if TRACE_LEVEL +#define TRACE(...) printf(__VA_ARGS__) +#else +#define TRACE(...) +#endif + +/* + * Port SOURCE + */ +#ifdef RTE_PORT_PCAP + +struct source { + struct { + struct rte_mempool *pool; + } params; + struct rte_swx_port_in_stats stats; + struct rte_mbuf **pkts; + uint32_t n_pkts; + uint32_t pos; +}; + +static void +source_free(void *port) +{ + struct source *p = port; + uint32_t i; + + if (!p) + return; + + for (i = 0; i < p->n_pkts; i++) + rte_pktmbuf_free(p->pkts[i]); + + free(p->pkts); + + free(p); +} + +static void * +source_create(void *args) +{ + char pcap_errbuf[PCAP_ERRBUF_SIZE]; + struct rte_swx_port_source_params *params = args; + struct source *p = NULL; + pcap_t *f = NULL; + uint32_t n_pkts_max, i; + + /* Check input arguments. */ + CHECK(params); + CHECK(params->pool); + CHECK(params->file_name && params->file_name[0]); + n_pkts_max = params->n_pkts_max ? + params->n_pkts_max : + RTE_SWX_PORT_SOURCE_PKTS_MAX; + + /* Resource allocation. */ + f = pcap_open_offline(params->file_name, pcap_errbuf); + if (!f) + goto error; + + p = calloc(1, sizeof(struct source)); + if (!p) + goto error; + + p->pkts = calloc(n_pkts_max, sizeof(struct rte_mbuf *)); + if (!p->pkts) + goto error; + + /* Initialization. */ + p->params.pool = params->pool; + + /* PCAP file. */ + for (i = 0; i < n_pkts_max; i++) { + struct pcap_pkthdr pcap_pkthdr; + const uint8_t *pcap_pktdata; + struct rte_mbuf *m; + uint8_t *m_data; + + /* Read new packet from PCAP file. */ + pcap_pktdata = pcap_next(f, &pcap_pkthdr); + if (!pcap_pktdata) + break; + + /* Allocate new buffer from pool. */ + m = rte_pktmbuf_alloc(params->pool); + if (!m) + goto error; + m_data = rte_pktmbuf_mtod(m, uint8_t *); + + rte_memcpy(m_data, pcap_pktdata, pcap_pkthdr.caplen); + m->data_len = pcap_pkthdr.caplen; + m->pkt_len = pcap_pkthdr.caplen; + + p->pkts[p->n_pkts] = m; + p->n_pkts++; + } + + if (!p->n_pkts) + goto error; + + pcap_close(f); + return p; + +error: + if (p) + source_free(p); + if (f) + pcap_close(f); + return NULL; +} + +static int +source_pkt_rx(void *port, struct rte_swx_pkt *pkt) +{ + struct source *p = port; + struct rte_mbuf *m_dst, *m_src; + uint8_t *m_dst_data, *m_src_data; + + /* m_src identification. */ + m_src = p->pkts[p->pos]; + m_src_data = rte_pktmbuf_mtod(m_src, uint8_t *); + + /* m_dst allocation from pool. */ + m_dst = rte_pktmbuf_alloc(p->params.pool); + if (!m_dst) + return 0; + + /* m_dst initialization. */ + m_dst->data_len = m_src->data_len; + m_dst->pkt_len = m_src->pkt_len; + m_dst->data_off = m_src->data_off; + + m_dst_data = rte_pktmbuf_mtod(m_dst, uint8_t *); + rte_memcpy(m_dst_data, m_src_data, m_src->data_len); + + /* pkt initialization. */ + pkt->handle = m_dst; + pkt->pkt = m_dst->buf_addr; + pkt->offset = m_dst->data_off; + pkt->length = m_dst->pkt_len; + + TRACE("[Source port] Pkt RX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + /* port stats update. */ + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + /* m_src next. */ + p->pos++; + if (p->pos == p->n_pkts) + p->pos = 0; + + return 1; +} + +static void +source_stats_read(void *port, struct rte_swx_port_in_stats *stats) +{ + struct source *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = source_create, + .free = source_free, + .pkt_rx = source_pkt_rx, + .stats_read = source_stats_read, +}; + +#else + +struct rte_swx_port_in_ops rte_swx_port_source_ops = { + .create = NULL, + .free = NULL, + .pkt_rx = NULL, + .stats_read = NULL, +}; + +#endif + +/* + * Port SINK + */ +#ifdef RTE_PORT_PCAP + +struct sink { + struct rte_swx_port_out_stats stats; + pcap_t *f_pcap; + pcap_dumper_t *f_dump; +}; + +static void +sink_free(void *port) +{ + struct sink *p = port; + + if (!p) + return; + + if (p->f_dump) + pcap_dump_close(p->f_dump); + if (p->f_pcap) + pcap_close(p->f_pcap); + free(p); +} + +static void * +sink_create(void *args) +{ + struct rte_swx_port_sink_params *params = args; + struct sink *p; + + /* Memory allocation. */ + p = calloc(1, sizeof(struct sink)); + if (!p) + goto error; + + if (params && params->file_name && params->file_name[0]) { + p->f_pcap = pcap_open_dead(DLT_EN10MB, 65535); + if (!p->f_pcap) + goto error; + + p->f_dump = pcap_dump_open(p->f_pcap, params->file_name); + if (!p->f_dump) + goto error; + } + + return p; + +error: + sink_free(p); + return NULL; +} + +static void +sink_pkt_tx(void *port, struct rte_swx_pkt *pkt) +{ + struct sink *p = port; + struct rte_mbuf *m = pkt->handle; + + TRACE("[Sink port] Pkt TX (%u bytes at offset %u)\n", + pkt->length, + pkt->offset); + if (TRACE_LEVEL) + rte_hexdump(stdout, NULL, &pkt->pkt[pkt->offset], pkt->length); + + m->pkt_len = pkt->length; + m->data_len = (uint16_t)pkt->length; + m->data_off = (uint16_t)pkt->offset; + + p->stats.n_pkts++; + p->stats.n_bytes += pkt->length; + + if (p->f_dump) { + struct pcap_pkthdr pcap_pkthdr; + uint8_t *m_data = rte_pktmbuf_mtod(m, uint8_t *); + + pcap_pkthdr.len = m->pkt_len; + pcap_pkthdr.caplen = m->data_len; + gettimeofday(&pcap_pkthdr.ts, NULL); + + pcap_dump((uint8_t *)p->f_dump, &pcap_pkthdr, m_data); + pcap_dump_flush(p->f_dump); + } + + rte_pktmbuf_free(m); +} + +static void +sink_stats_read(void *port, struct rte_swx_port_out_stats *stats) +{ + struct sink *p = port; + + if (!p || !stats) + return; + + memcpy(stats, &p->stats, sizeof(p->stats)); +} + +/* + * Summary of port operations + */ +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = sink_create, + .free = sink_free, + .pkt_tx = sink_pkt_tx, + .flush = NULL, + .stats_read = sink_stats_read, +}; + +#else + +struct rte_swx_port_out_ops rte_swx_port_sink_ops = { + .create = NULL, + .free = NULL, + .pkt_tx = NULL, + .flush = NULL, + .stats_read = NULL, +}; + +#endif diff --git a/lib/librte_port/rte_swx_port_source_sink.h b/lib/librte_port/rte_swx_port_source_sink.h new file mode 100644 index 000000000..88a890c5a --- /dev/null +++ b/lib/librte_port/rte_swx_port_source_sink.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ +#define __INCLUDE_RTE_SWX_PORT_SOURCE_SINK_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Source and Sink Ports + */ + +#include "rte_swx_port.h" + +/** Maximum number of packets to read from the PCAP file. */ +#ifndef RTE_SWX_PORT_SOURCE_PKTS_MAX +#define RTE_SWX_PORT_SOURCE_PKTS_MAX 1024 +#endif + +/** Source port creation parameters. */ +struct rte_swx_port_source_params { + /** Buffer pool. Must be valid. */ + struct rte_mempool *pool; + + /** Name of a valid PCAP file to read the input packets from. */ + const char *file_name; + + /** Maximum number of packets to read from the PCAP file. When 0, it is + * internally set to RTE_SWX_PORT_SOURCE_PKTS_MAX. Once read from the + * PCAP file, the same packets are looped forever. + */ + uint32_t n_pkts_max; +}; + +/** Source port operations. */ +extern struct rte_swx_port_in_ops rte_swx_port_source_ops; + +/** Sink port creation parameters. */ +struct rte_swx_port_sink_params { + /** Name of a valid PCAP file to write the output packets to. When NULL, + * all the output packets are dropped instead of being saved to a PCAP + * file. + */ + const char *file_name; +}; + +/** Sink port operations. */ +extern struct rte_swx_port_out_ops rte_swx_port_sink_ops; + +#ifdef __cplusplus +} +#endif + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 34/40] table: add exact match table 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (32 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 33/40] port: add source and sink ports Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 35/40] examples/pipeline: add new example application Cristian Dumitrescu ` (5 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add the exact match pipeline table type. Used under the hood by the pipeline table instruction. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- lib/librte_table/Makefile | 2 + lib/librte_table/meson.build | 6 +- lib/librte_table/rte_swx_table_em.c | 851 +++++++++++++++++++++++++ lib/librte_table/rte_swx_table_em.h | 30 + lib/librte_table/rte_table_version.map | 7 + 5 files changed, 894 insertions(+), 2 deletions(-) create mode 100644 lib/librte_table/rte_swx_table_em.c create mode 100644 lib/librte_table/rte_swx_table_em.h diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 9df58698d..f544fd5af 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -34,6 +34,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_ext.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_hash_lru.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_array.c SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_table_stub.c +SRCS-$(CONFIG_RTE_LIBRTE_TABLE) += rte_swx_table_em.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table.h @@ -56,5 +57,6 @@ endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_swx_table_em.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/meson.build b/lib/librte_table/meson.build index b9d4fe3dc..d69678386 100644 --- a/lib/librte_table/meson.build +++ b/lib/librte_table/meson.build @@ -11,7 +11,8 @@ sources = files('rte_table_acl.c', 'rte_table_hash_ext.c', 'rte_table_hash_lru.c', 'rte_table_array.c', - 'rte_table_stub.c') + 'rte_table_stub.c', + 'rte_swx_table_em.c',) headers = files('rte_table.h', 'rte_table_acl.h', 'rte_table_lpm.h', @@ -23,7 +24,8 @@ headers = files('rte_table.h', 'rte_lru.h', 'rte_table_array.h', 'rte_table_stub.h', - 'rte_swx_table.h',) + 'rte_swx_table.h', + 'rte_swx_table_em.h',) deps += ['mbuf', 'port', 'lpm', 'hash', 'acl'] if arch_subdir == 'x86' diff --git a/lib/librte_table/rte_swx_table_em.c b/lib/librte_table/rte_swx_table_em.c new file mode 100644 index 000000000..85c77ad03 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.c @@ -0,0 +1,851 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <errno.h> + +#include <rte_common.h> +#include <rte_prefetch.h> + +#include "rte_swx_table_em.h" + +#define CHECK(condition, err_code) \ +do { \ + if (!(condition)) \ + return -(err_code); \ +} while (0) + +#ifndef RTE_SWX_TABLE_EM_USE_HUGE_PAGES +#define RTE_SWX_TABLE_EM_USE_HUGE_PAGES 1 +#endif + +#if RTE_SWX_TABLE_EM_USE_HUGE_PAGES + +#include <rte_malloc.h> + +static void * +env_malloc(size_t size, size_t alignment, int numa_node) +{ + return rte_zmalloc_socket(NULL, size, alignment, numa_node); +} + +static void +env_free(void *start, size_t size __rte_unused) +{ + rte_free(start); +} + +#else + +#include <numa.h> + +static void * +env_malloc(size_t size, size_t alignment __rte_unused, int numa_node) +{ + return numa_alloc_onnode(size, numa_node); +} + +static void +env_free(void *start, size_t size) +{ + numa_free(start, size); +} + +#endif + +#if defined(RTE_ARCH_X86_64) + +#include <x86intrin.h> + +#define crc32_u64(crc, v) _mm_crc32_u64(crc, v) + +#else + +static inline uint64_t +crc32_u64_generic(uint64_t crc, uint64_t value) +{ + int i; + + crc = (crc & 0xFFFFFFFFLLU) ^ value; + for (i = 63; i >= 0; i--) { + uint64_t mask; + + mask = -(crc & 1LLU); + crc = (crc >> 1LLU) ^ (0x82F63B78LLU & mask); + } + + return crc; +} + +#define crc32_u64(crc, v) crc32_u64_generic(crc, v) + +#endif + +/* Key size needs to be one of: 8, 16, 32 or 64. */ +static inline uint32_t +hash(void *key, void *key_mask, uint32_t key_size, uint32_t seed) +{ + uint64_t *k = key; + uint64_t *m = key_mask; + uint64_t k0, k2, k5, crc0, crc1, crc2, crc3, crc4, crc5; + + switch (key_size) { + case 8: + crc0 = crc32_u64(seed, k[0] & m[0]); + return crc0; + + case 16: + k0 = k[0] & m[0]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc0 ^= crc1; + + return crc0; + + case 32: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = k2 >> 32; + + crc0 = crc32_u64(crc0, crc1); + crc1 = crc32_u64(crc2, crc3); + + crc0 ^= crc1; + + return crc0; + + case 64: + k0 = k[0] & m[0]; + k2 = k[2] & m[2]; + k5 = k[5] & m[5]; + + crc0 = crc32_u64(k0, seed); + crc1 = crc32_u64(k0 >> 32, k[1] & m[1]); + + crc2 = crc32_u64(k2, k[3] & m[3]); + crc3 = crc32_u64(k2 >> 32, k[4] & m[4]); + + crc4 = crc32_u64(k5, k[6] & m[6]); + crc5 = crc32_u64(k5 >> 32, k[7] & m[7]); + + crc0 = crc32_u64(crc0, (crc1 << 32) ^ crc2); + crc1 = crc32_u64(crc3, (crc4 << 32) ^ crc5); + + crc0 ^= crc1; + + return crc0; + + default: + crc0 = 0; + return crc0; + } +} + +/* n_bytes needs to be a multiple of 8 bytes. */ +static void +keycpy(void *dst, void *src, void *src_mask, uint32_t n_bytes) +{ + uint64_t *dst64 = dst, *src64 = src, *src_mask64 = src_mask; + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + dst64[i] = src64[i] & src_mask64[i]; +} + +/* + * Return: 0 = Keys are NOT equal; 1 = Keys are equal. + */ +static inline uint32_t +keycmp(void *a, void *b, void *b_mask, uint32_t n_bytes) +{ + uint64_t *a64 = a, *b64 = b, *b_mask64 = b_mask; + + switch (n_bytes) { + case 8: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint32_t result = 1; + + if (xor0) + result = 0; + return result; + } + + case 16: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t or = xor0 | xor1; + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 32: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t or = (xor0 | xor1) | (xor2 | xor3); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + case 64: { + uint64_t xor0 = a64[0] ^ (b64[0] & b_mask64[0]); + uint64_t xor1 = a64[1] ^ (b64[1] & b_mask64[1]); + uint64_t xor2 = a64[2] ^ (b64[2] & b_mask64[2]); + uint64_t xor3 = a64[3] ^ (b64[3] & b_mask64[3]); + uint64_t xor4 = a64[4] ^ (b64[4] & b_mask64[4]); + uint64_t xor5 = a64[5] ^ (b64[5] & b_mask64[5]); + uint64_t xor6 = a64[6] ^ (b64[6] & b_mask64[6]); + uint64_t xor7 = a64[7] ^ (b64[7] & b_mask64[7]); + uint64_t or = ((xor0 | xor1) | (xor2 | xor3)) | + ((xor4 | xor5) | (xor6 | xor7)); + uint32_t result = 1; + + if (or) + result = 0; + return result; + } + + default: { + uint32_t i; + + for (i = 0; i < n_bytes / sizeof(uint64_t); i++) + if (a64[i] != (b64[i] & b_mask64[i])) + return 0; + return 1; + } + } +} + +#define KEYS_PER_BUCKET 4 + +struct bucket_extension { + struct bucket_extension *next; + uint16_t sig[KEYS_PER_BUCKET]; + uint32_t key_id[KEYS_PER_BUCKET]; +}; + +struct table { + /* Input parameters */ + struct rte_swx_table_params params; + + /* Internal. */ + uint32_t key_size; + uint32_t data_size; + uint32_t key_size_shl; + uint32_t data_size_shl; + uint32_t n_buckets; + uint32_t n_buckets_ext; + uint32_t key_stack_tos; + uint32_t bkt_ext_stack_tos; + uint64_t total_size; + + /* Memory arrays. */ + uint8_t *key_mask; + struct bucket_extension *buckets; + struct bucket_extension *buckets_ext; + uint8_t *keys; + uint32_t *key_stack; + uint32_t *bkt_ext_stack; + uint8_t *data; +}; + +static inline uint8_t * +table_key(struct table *t, uint32_t key_id) +{ + return &t->keys[(uint64_t)key_id << t->key_size_shl]; +} + +static inline uint64_t * +table_key_data(struct table *t, uint32_t key_id) +{ + return (uint64_t *)&t->data[(uint64_t)key_id << t->data_size_shl]; +} + +static inline int +bkt_is_empty(struct bucket_extension *bkt) +{ + return (!bkt->sig[0] && !bkt->sig[1] && !bkt->sig[2] && !bkt->sig[2]) ? + 1 : 0; +} + +/* Return: + * 0 = Bucket key position is NOT empty; + * 1 = Bucket key position is empty. + */ +static inline int +bkt_key_is_empty(struct bucket_extension *bkt, uint32_t bkt_pos) +{ + return bkt->sig[bkt_pos] ? 0 : 1; +} + +/* Return: 0 = Keys are NOT equal; 1 = Keys are equal. */ +static inline int +bkt_keycmp(struct table *t, + struct bucket_extension *bkt, + uint8_t *input_key, + uint32_t bkt_pos, + uint32_t input_sig) +{ + uint32_t bkt_key_id; + uint8_t *bkt_key; + + /* Key signature comparison. */ + if (input_sig != bkt->sig[bkt_pos]) + return 0; + + /* Key comparison. */ + bkt_key_id = bkt->key_id[bkt_pos]; + bkt_key = table_key(t, bkt_key_id); + return keycmp(bkt_key, input_key, t->key_mask, t->key_size); +} + +static inline void +bkt_key_install(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos, + uint32_t bkt_key_id, + uint32_t input_sig) +{ + uint8_t *bkt_key; + uint64_t *bkt_data; + + /* Key signature. */ + bkt->sig[bkt_pos] = (uint16_t)input_sig; + + /* Key. */ + bkt->key_id[bkt_pos] = bkt_key_id; + bkt_key = table_key(t, bkt_key_id); + keycpy(bkt_key, input->key, t->key_mask, t->key_size); + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +static inline void +bkt_key_data_update(struct table *t, + struct bucket_extension *bkt, + struct rte_swx_table_entry *input, + uint32_t bkt_pos) +{ + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[bkt_pos]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + bkt_data[0] = input->action_id; + if (t->params.action_data_size) + memcpy(&bkt_data[1], + input->action_data, + t->params.action_data_size); +} + +#define CL RTE_CACHE_LINE_ROUNDUP + +static int +__table_create(struct table **table, + uint64_t *memory_footprint, + struct rte_swx_table_params *params, + const char *args __rte_unused, + int numa_node) +{ + struct table *t; + uint8_t *memory; + size_t table_meta_sz, key_mask_sz, bucket_sz, bucket_ext_sz, key_sz, + key_stack_sz, bkt_ext_stack_sz, data_sz, total_size; + size_t key_mask_offset, bucket_offset, bucket_ext_offset, key_offset, + key_stack_offset, bkt_ext_stack_offset, data_offset; + uint32_t key_size, key_data_size, n_buckets, n_buckets_ext, i; + + /* Check input arguments. */ + CHECK(params, EINVAL); + CHECK(params->match_type == RTE_SWX_TABLE_MATCH_EXACT, EINVAL); + CHECK(params->key_size, EINVAL); + CHECK(params->key_size <= 64, EINVAL); + CHECK(params->n_keys_max, EINVAL); + + /* Memory allocation. */ + key_size = rte_align64pow2(params->key_size); + if (key_size < 8) + key_size = 8; + key_data_size = rte_align64pow2(params->action_data_size + 8); + n_buckets = params->n_keys_max / KEYS_PER_BUCKET; + n_buckets_ext = params->n_keys_max / KEYS_PER_BUCKET; + + table_meta_sz = CL(sizeof(struct table)); + key_mask_sz = CL(key_size); + bucket_sz = CL(n_buckets * sizeof(struct bucket_extension)); + bucket_ext_sz = CL(n_buckets_ext * sizeof(struct bucket_extension)); + key_sz = CL(params->n_keys_max * key_size); + key_stack_sz = CL(params->n_keys_max * sizeof(uint32_t)); + bkt_ext_stack_sz = CL(n_buckets_ext * sizeof(uint32_t)); + data_sz = CL(params->n_keys_max * key_data_size); + total_size = table_meta_sz + key_mask_sz + bucket_sz + bucket_ext_sz + + key_sz + key_stack_sz + bkt_ext_stack_sz + data_sz; + + key_mask_offset = table_meta_sz; + bucket_offset = key_mask_offset + key_mask_sz; + bucket_ext_offset = bucket_offset + bucket_sz; + key_offset = bucket_ext_offset + bucket_ext_sz; + key_stack_offset = key_offset + key_sz; + bkt_ext_stack_offset = key_stack_offset + key_stack_sz; + data_offset = bkt_ext_stack_offset + bkt_ext_stack_sz; + + if (!table) { + if (memory_footprint) + *memory_footprint = total_size; + return 0; + } + + memory = env_malloc(total_size, RTE_CACHE_LINE_SIZE, numa_node); + CHECK(memory, ENOMEM); + memset(memory, 0, total_size); + + /* Initialization. */ + t = (struct table *)memory; + memcpy(&t->params, params, sizeof(*params)); + + t->key_size = key_size; + t->data_size = key_data_size; + t->key_size_shl = __builtin_ctzl(key_size); + t->data_size_shl = __builtin_ctzl(key_data_size); + t->n_buckets = n_buckets; + t->n_buckets_ext = n_buckets_ext; + t->total_size = total_size; + + t->key_mask = &memory[key_mask_offset]; + t->buckets = (struct bucket_extension *)&memory[bucket_offset]; + t->buckets_ext = (struct bucket_extension *)&memory[bucket_ext_offset]; + t->keys = &memory[key_offset]; + t->key_stack = (uint32_t *)&memory[key_stack_offset]; + t->bkt_ext_stack = (uint32_t *)&memory[bkt_ext_stack_offset]; + t->data = &memory[data_offset]; + + t->params.key_mask0 = t->key_mask; + + if (!params->key_mask0) + memset(t->key_mask, 0xFF, params->key_size); + else + memcpy(t->key_mask, params->key_mask0, params->key_size); + + for (i = 0; i < t->params.n_keys_max; i++) + t->key_stack[i] = t->params.n_keys_max - 1 - i; + t->key_stack_tos = t->params.n_keys_max; + + for (i = 0; i < n_buckets_ext; i++) + t->bkt_ext_stack[i] = n_buckets_ext - 1 - i; + t->bkt_ext_stack_tos = n_buckets_ext; + + *table = t; + return 0; +} + +static void +table_free(void *table) +{ + struct table *t = table; + + if (!t) + return; + + env_free(t, t->total_size); +} + +static int +table_add(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + CHECK((!t->params.action_data_size && !entry->action_data) || + (t->params.action_data_size && entry->action_data), EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + bkt_key_data_update(t, bkt, entry, i); + return 0; + } + + /* Key is not present in the bucket. Bucket not full. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_key_is_empty(bkt, i)) { + uint32_t new_bkt_key_id; + + /* Allocate new key & install. */ + CHECK(t->key_stack_tos, ENOSPC); + new_bkt_key_id = + t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, bkt, entry, i, + new_bkt_key_id, input_sig); + return 0; + } + + /* Bucket full: extend bucket. */ + if (t->bkt_ext_stack_tos && t->key_stack_tos) { + struct bucket_extension *new_bkt; + uint32_t new_bkt_id, new_bkt_key_id; + + /* Allocate new bucket extension & install. */ + new_bkt_id = t->bkt_ext_stack[--t->bkt_ext_stack_tos]; + new_bkt = &t->buckets_ext[new_bkt_id]; + memset(new_bkt, 0, sizeof(*new_bkt)); + bkt_prev->next = new_bkt; + + /* Allocate new key & install. */ + new_bkt_key_id = t->key_stack[--t->key_stack_tos]; + bkt_key_install(t, new_bkt, entry, 0, + new_bkt_key_id, input_sig); + return 0; + } + + CHECK(0, ENOSPC); +} + +static int +table_del(void *table, struct rte_swx_table_entry *entry) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt, *bkt_prev; + uint32_t input_sig, bkt_id, i; + + CHECK(t, EINVAL); + CHECK(entry, EINVAL); + CHECK(entry->key, EINVAL); + + input_sig = hash(entry->key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0, bkt_prev = NULL; bkt; bkt_prev = bkt, bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, entry->key, i, input_sig)) { + /* Key free. */ + bkt->sig[i] = 0; + t->key_stack[t->key_stack_tos++] = + bkt->key_id[i]; + + /* Bucket extension free if empty and not the + * 1st in bucket. + */ + if (bkt_prev && bkt_is_empty(bkt)) { + bkt_prev->next = bkt->next; + bkt_id = bkt - t->buckets_ext; + t->bkt_ext_stack[t->bkt_ext_stack_tos++] + = bkt_id; + } + + return 0; + } + + return 0; +} + +static uint64_t +table_mailbox_size_get_unoptimized(void) +{ + return 0; +} + +static int +table_lookup_unoptimized(void *table, + void *mailbox __rte_unused, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct bucket_extension *bkt0, *bkt; + uint8_t *input_key; + uint32_t input_sig, bkt_id, i; + + input_key = &(*key)[t->params.key_offset]; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt0 = &t->buckets[bkt_id]; + input_sig = (input_sig >> 16) | 1; + + /* Key is present in the bucket. */ + for (bkt = bkt0; bkt; bkt = bkt->next) + for (i = 0; i < KEYS_PER_BUCKET; i++) + if (bkt_keycmp(t, bkt, input_key, i, input_sig)) { + uint32_t bkt_key_id; + uint64_t *bkt_data; + + /* Key. */ + bkt_key_id = bkt->key_id[i]; + + /* Key data. */ + bkt_data = table_key_data(t, bkt_key_id); + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = 1; + return 1; + } + + *hit = 0; + return 1; +} + +struct mailbox { + struct bucket_extension *bkt; + uint32_t input_sig; + uint32_t bkt_key_id; + uint32_t sig_match; + uint32_t sig_match_many; + int state; +}; + +static uint64_t +table_mailbox_size_get(void) +{ + return sizeof(struct mailbox); +} + +/* + * mask = match bitmask + * match = at least one match + * match_many = more than one match + * match_pos = position of first match + * + *+------+-------+------------+-----------+ + *| mask | match | match_many | match_pos | + *+------+-------+------------+-----------+ + *| 0000 | 0 | 0 | 00 | + *| 0001 | 1 | 0 | 00 | + *| 0010 | 1 | 0 | 01 | + *| 0011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 0100 | 1 | 0 | 10 | + *| 0101 | 1 | 1 | 00 | + *| 0110 | 1 | 1 | 01 | + *| 0111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1000 | 1 | 0 | 11 | + *| 1001 | 1 | 1 | 00 | + *| 1010 | 1 | 1 | 01 | + *| 1011 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + *| 1100 | 1 | 1 | 10 | + *| 1101 | 1 | 1 | 00 | + *| 1110 | 1 | 1 | 01 | + *| 1111 | 1 | 1 | 00 | + *+------+-------+------------+-----------+ + * + * match = 1111_1111_1111_1110 = 0xFFFE + * match_many = 1111_1110_1110_1000 = 0xFEE8 + * match_pos = 0001_0010_0001_0011__0001_0010_0001_0000 = 0x12131210 + * + */ + +#define LUT_MATCH 0xFFFE +#define LUT_MATCH_MANY 0xFEE8 +#define LUT_MATCH_POS 0x12131210 + +static int +table_lookup(void *table, + void *mailbox, + uint8_t **key, + uint64_t *action_id, + uint8_t **action_data, + int *hit) +{ + struct table *t = table; + struct mailbox *m = mailbox; + + switch (m->state) { + case 0: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt; + uint32_t input_sig, bkt_id; + + input_sig = hash(input_key, t->key_mask, t->key_size, 0); + bkt_id = input_sig & (t->n_buckets - 1); + bkt = &t->buckets[bkt_id]; + rte_prefetch0(bkt); + + m->bkt = bkt; + m->input_sig = (input_sig >> 16) | 1; + m->state++; + return 0; + } + + case 1: { + struct bucket_extension *bkt = m->bkt; + uint32_t input_sig = m->input_sig; + uint32_t bkt_sig0, bkt_sig1, bkt_sig2, bkt_sig3; + uint32_t mask0 = 0, mask1 = 0, mask2 = 0, mask3 = 0, mask_all; + uint32_t sig_match = LUT_MATCH; + uint32_t sig_match_many = LUT_MATCH_MANY; + uint32_t sig_match_pos = LUT_MATCH_POS; + uint32_t bkt_key_id; + + bkt_sig0 = input_sig ^ bkt->sig[0]; + if (bkt_sig0) + mask0 = 1 << 0; + + bkt_sig1 = input_sig ^ bkt->sig[1]; + if (bkt_sig1) + mask1 = 1 << 1; + + bkt_sig2 = input_sig ^ bkt->sig[2]; + if (bkt_sig2) + mask2 = 1 << 2; + + bkt_sig3 = input_sig ^ bkt->sig[3]; + if (bkt_sig3) + mask3 = 1 << 3; + + mask_all = (mask0 | mask1) | (mask2 | mask3); + sig_match = (sig_match >> mask_all) & 1; + sig_match_many = (sig_match_many >> mask_all) & 1; + sig_match_pos = (sig_match_pos >> (mask_all << 1)) & 3; + + bkt_key_id = bkt->key_id[sig_match_pos]; + rte_prefetch0(table_key(t, bkt_key_id)); + rte_prefetch0(table_key_data(t, bkt_key_id)); + + m->bkt_key_id = bkt_key_id; + m->sig_match = sig_match; + m->sig_match_many = sig_match_many; + m->state++; + return 0; + } + + case 2: { + uint8_t *input_key = &(*key)[t->params.key_offset]; + struct bucket_extension *bkt = m->bkt; + uint32_t bkt_key_id = m->bkt_key_id; + uint8_t *bkt_key = table_key(t, bkt_key_id); + uint64_t *bkt_data = table_key_data(t, bkt_key_id); + uint32_t lkp_hit; + + lkp_hit = keycmp(bkt_key, input_key, t->key_mask, t->key_size); + lkp_hit &= m->sig_match; + *action_id = bkt_data[0]; + *action_data = (uint8_t *)&bkt_data[1]; + *hit = lkp_hit; + + m->state = 0; + + if (!lkp_hit && (m->sig_match_many || bkt->next)) + return table_lookup_unoptimized(t, + m, + key, + action_id, + action_data, + hit); + + return 1; + } + + default: + return 0; + } +} + +static void * +table_create(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries, + const char *args, + int numa_node) +{ + struct table *t; + struct rte_swx_table_entry *entry; + int status; + + /* Table create. */ + status = __table_create(&t, NULL, params, args, numa_node); + if (status) + return NULL; + + /* Table add entries. */ + if (!entries) + return t; + + TAILQ_FOREACH(entry, entries, node) { + int status; + + status = table_add(t, entry); + if (status) { + table_free(t); + return NULL; + } + } + + return t; +} + +static uint64_t +table_footprint(struct rte_swx_table_params *params, + struct rte_swx_table_entry_list *entries __rte_unused, + const char *args) +{ + uint64_t memory_footprint; + int status; + + status = __table_create(NULL, &memory_footprint, params, args, 0); + if (status) + return 0; + + return memory_footprint; +} + +struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get_unoptimized, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup_unoptimized, + .free = table_free, +}; + +struct rte_swx_table_ops rte_swx_table_exact_match_ops = { + .footprint_get = table_footprint, + .mailbox_size_get = table_mailbox_size_get, + .create = table_create, + .add = table_add, + .del = table_del, + .lkp = table_lookup, + .free = table_free, +}; diff --git a/lib/librte_table/rte_swx_table_em.h b/lib/librte_table/rte_swx_table_em.h new file mode 100644 index 000000000..909ada483 --- /dev/null +++ b/lib/librte_table/rte_swx_table_em.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#ifndef __INCLUDE_RTE_SWX_TABLE_EM_H__ +#define __INCLUDE_RTE_SWX_TABLE_EM_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * @file + * RTE SWX Exact Match Table + */ + +#include <stdint.h> + +#include <rte_swx_table.h> + +/** Exact match table operations - unoptimized. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_unoptimized_ops; + +/** Exact match table operations. */ +extern struct rte_swx_table_ops rte_swx_table_exact_match_ops; + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/librte_table/rte_table_version.map b/lib/librte_table/rte_table_version.map index 568a6c6a8..81c554b63 100644 --- a/lib/librte_table/rte_table_version.map +++ b/lib/librte_table/rte_table_version.map @@ -18,3 +18,10 @@ DPDK_21 { local: *; }; + +EXPERIMENTAL { + global: + + rte_swx_table_exact_match_unoptimized_ops; + rte_swx_table_exact_match_ops; +}; -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 35/40] examples/pipeline: add new example application 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (33 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 34/40] table: add exact match table Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 36/40] examples/pipeline: add message passing mechanism Cristian Dumitrescu ` (4 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add new example application to showcase the new pipeline type API. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/Makefile | 1 + examples/meson.build | 1 + examples/pipeline/Makefile | 80 +++++ examples/pipeline/main.c | 52 ++++ examples/pipeline/meson.build | 16 + examples/pipeline/obj.c | 470 +++++++++++++++++++++++++++++ examples/pipeline/obj.h | 131 ++++++++ examples/pipeline/thread.c | 549 ++++++++++++++++++++++++++++++++++ examples/pipeline/thread.h | 28 ++ 9 files changed, 1328 insertions(+) create mode 100644 examples/pipeline/Makefile create mode 100644 examples/pipeline/main.c create mode 100644 examples/pipeline/meson.build create mode 100644 examples/pipeline/obj.c create mode 100644 examples/pipeline/obj.h create mode 100644 examples/pipeline/thread.c create mode 100644 examples/pipeline/thread.h diff --git a/examples/Makefile b/examples/Makefile index b7e99a2f7..b1ebac681 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -61,6 +61,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += packet_ordering ifeq ($(CONFIG_RTE_ARCH_X86_64),y) DIRS-y += performance-thread endif +DIRS-$(CONFIG_RTE_LIBRTE_PIPELINE) += pipeline DIRS-$(CONFIG_RTE_LIBRTE_IEEE1588) += ptpclient DIRS-$(CONFIG_RTE_LIBRTE_METER) += qos_meter DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += qos_sched diff --git a/examples/meson.build b/examples/meson.build index eb13e8210..245d98575 100644 --- a/examples/meson.build +++ b/examples/meson.build @@ -33,6 +33,7 @@ all_examples = [ 'ntb', 'packet_ordering', 'performance-thread/l3fwd-thread', 'performance-thread/pthread_shim', + 'pipeline', 'ptpclient', 'qos_meter', 'qos_sched', 'rxtx_callbacks', diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile new file mode 100644 index 000000000..3c85e9e40 --- /dev/null +++ b/examples/pipeline/Makefile @@ -0,0 +1,80 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2020 Intel Corporation + +# binary name +APP = pipeline + +# all source are stored in SRCS-y +SRCS-y += main.c +SRCS-y += obj.c +SRCS-y += thread.c + +# Build using pkg-config variables if possible +ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) + +all: shared +.PHONY: shared static +shared: build/$(APP)-shared + ln -sf $(APP)-shared build/$(APP) +static: build/$(APP)-static + ln -sf $(APP)-static build/$(APP) + +PKGCONF ?= pkg-config + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk) +LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) +LDFLAGS_STATIC = -Wl,-Bstatic $(shell $(PKGCONF) --static --libs libdpdk) + +CFLAGS += -I. + +OBJS := $(patsubst %.c,build/%.o,$(SRCS-y)) + +build/%.o: %.c Makefile $(PC_FILE) | build + $(CC) $(CFLAGS) -c $< -o $@ + +build/$(APP)-shared: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) + +build/$(APP)-static: $(OBJS) + $(CC) $(OBJS) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC) + +build: + @mkdir -p $@ + +.PHONY: clean +clean: + rm -f build/$(APP)* build/*.o + test -d build && rmdir -p build || true + +else + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, detect a build directory, by looking for a path with a .config +RTE_TARGET ?= $(notdir $(abspath $(dir $(firstword $(wildcard $(RTE_SDK)/*/.config))))) + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUX),y) +$(info This application can only operate in a linux environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +clean: +else + +INC += $(sort $(wildcard *.h)) + +SRCS-$(CONFIG_RTE_LIBRTE_PIPELINE) := $(SRCS-y) + +CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(SRCDIR) +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif +endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c new file mode 100644 index 000000000..353e62c10 --- /dev/null +++ b/examples/pipeline/main.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <string.h> +#include <fcntl.h> +#include <unistd.h> +#include <getopt.h> + +#include <rte_launch.h> +#include <rte_eal.h> + +#include "obj.h" +#include "thread.h" + +int +main(int argc, char **argv) +{ + struct obj *obj; + int status; + + /* EAL */ + status = rte_eal_init(argc, argv); + if (status < 0) { + printf("Error: EAL initialization failed (%d)\n", status); + return status; + }; + + /* Obj */ + obj = obj_init(); + if (!obj) { + printf("Error: Obj initialization failed (%d)\n", status); + return status; + } + + /* Thread */ + status = thread_init(); + if (status) { + printf("Error: Thread initialization failed (%d)\n", status); + return status; + } + + rte_eal_mp_remote_launch( + thread_main, + NULL, + SKIP_MASTER); + + /* Dispatch loop */ + for ( ; ; ); +} + diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build new file mode 100644 index 000000000..ade485f97 --- /dev/null +++ b/examples/pipeline/meson.build @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2017-2020 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +build = cc.has_header('sys/epoll.h') +deps += ['pipeline', 'bus_pci'] +allow_experimental_apis = true +sources = files( + 'main.c', + 'obj.c', + 'thread.c', +) diff --git a/examples/pipeline/obj.c b/examples/pipeline/obj.c new file mode 100644 index 000000000..e37e60540 --- /dev/null +++ b/examples/pipeline/obj.c @@ -0,0 +1,470 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> +#include <string.h> + +#include <rte_mempool.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "obj.h" + +/* + * mempool + */ +TAILQ_HEAD(mempool_list, mempool); + +/* + * link + */ +TAILQ_HEAD(link_list, link); + +/* + * pipeline + */ +TAILQ_HEAD(pipeline_list, pipeline); + +/* + * obj + */ +struct obj { + struct mempool_list mempool_list; + struct link_list link_list; + struct pipeline_list pipeline_list; +}; + +/* + * mempool + */ +#define BUFFER_SIZE_MIN (sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM) + +struct mempool * +mempool_create(struct obj *obj, const char *name, struct mempool_params *params) +{ + struct mempool *mempool; + struct rte_mempool *m; + + /* Check input params */ + if ((name == NULL) || + mempool_find(obj, name) || + (params == NULL) || + (params->buffer_size < BUFFER_SIZE_MIN) || + (params->pool_size == 0)) + return NULL; + + /* Resource create */ + m = rte_pktmbuf_pool_create( + name, + params->pool_size, + params->cache_size, + 0, + params->buffer_size - sizeof(struct rte_mbuf), + params->cpu_id); + + if (m == NULL) + return NULL; + + /* Node allocation */ + mempool = calloc(1, sizeof(struct mempool)); + if (mempool == NULL) { + rte_mempool_free(m); + return NULL; + } + + /* Node fill in */ + strlcpy(mempool->name, name, sizeof(mempool->name)); + mempool->m = m; + mempool->buffer_size = params->buffer_size; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->mempool_list, mempool, node); + + return mempool; +} + +struct mempool * +mempool_find(struct obj *obj, const char *name) +{ + struct mempool *mempool; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(mempool, &obj->mempool_list, node) + if (strcmp(mempool->name, name) == 0) + return mempool; + + return NULL; +} + +/* + * link + */ +static struct rte_eth_conf port_conf_default = { + .link_speeds = 0, + .rxmode = { + .mq_mode = ETH_MQ_RX_NONE, + .max_rx_pkt_len = 9000, /* Jumbo frame max packet len */ + .split_hdr_size = 0, /* Header split buffer size */ + }, + .rx_adv_conf = { + .rss_conf = { + .rss_key = NULL, + .rss_key_len = 40, + .rss_hf = 0, + }, + }, + .txmode = { + .mq_mode = ETH_MQ_TX_NONE, + }, + .lpbk_mode = 0, +}; + +#define RETA_CONF_SIZE (ETH_RSS_RETA_SIZE_512 / RTE_RETA_GROUP_SIZE) + +static int +rss_setup(uint16_t port_id, + uint16_t reta_size, + struct link_params_rss *rss) +{ + struct rte_eth_rss_reta_entry64 reta_conf[RETA_CONF_SIZE]; + uint32_t i; + int status; + + /* RETA setting */ + memset(reta_conf, 0, sizeof(reta_conf)); + + for (i = 0; i < reta_size; i++) + reta_conf[i / RTE_RETA_GROUP_SIZE].mask = UINT64_MAX; + + for (i = 0; i < reta_size; i++) { + uint32_t reta_id = i / RTE_RETA_GROUP_SIZE; + uint32_t reta_pos = i % RTE_RETA_GROUP_SIZE; + uint32_t rss_qs_pos = i % rss->n_queues; + + reta_conf[reta_id].reta[reta_pos] = + (uint16_t) rss->queue_id[rss_qs_pos]; + } + + /* RETA update */ + status = rte_eth_dev_rss_reta_update(port_id, + reta_conf, + reta_size); + + return status; +} + +struct link * +link_create(struct obj *obj, const char *name, struct link_params *params) +{ + struct rte_eth_dev_info port_info; + struct rte_eth_conf port_conf; + struct link *link; + struct link_params_rss *rss; + struct mempool *mempool; + uint32_t cpu_id, i; + int status; + uint16_t port_id; + + /* Check input params */ + if ((name == NULL) || + link_find(obj, name) || + (params == NULL) || + (params->rx.n_queues == 0) || + (params->rx.queue_size == 0) || + (params->tx.n_queues == 0) || + (params->tx.queue_size == 0)) + return NULL; + + port_id = params->port_id; + if (params->dev_name) { + status = rte_eth_dev_get_port_by_name(params->dev_name, + &port_id); + + if (status) + return NULL; + } else + if (!rte_eth_dev_is_valid_port(port_id)) + return NULL; + + if (rte_eth_dev_info_get(port_id, &port_info) != 0) + return NULL; + + mempool = mempool_find(obj, params->rx.mempool_name); + if (mempool == NULL) + return NULL; + + rss = params->rx.rss; + if (rss) { + if ((port_info.reta_size == 0) || + (port_info.reta_size > ETH_RSS_RETA_SIZE_512)) + return NULL; + + if ((rss->n_queues == 0) || + (rss->n_queues >= LINK_RXQ_RSS_MAX)) + return NULL; + + for (i = 0; i < rss->n_queues; i++) + if (rss->queue_id[i] >= port_info.max_rx_queues) + return NULL; + } + + /** + * Resource create + */ + /* Port */ + memcpy(&port_conf, &port_conf_default, sizeof(port_conf)); + if (rss) { + port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; + port_conf.rx_adv_conf.rss_conf.rss_hf = + (ETH_RSS_IP | ETH_RSS_TCP | ETH_RSS_UDP) & + port_info.flow_type_rss_offloads; + } + + cpu_id = (uint32_t) rte_eth_dev_socket_id(port_id); + if (cpu_id == (uint32_t) SOCKET_ID_ANY) + cpu_id = 0; + + status = rte_eth_dev_configure( + port_id, + params->rx.n_queues, + params->tx.n_queues, + &port_conf); + + if (status < 0) + return NULL; + + if (params->promiscuous) { + status = rte_eth_promiscuous_enable(port_id); + if (status != 0) + return NULL; + } + + /* Port RX */ + for (i = 0; i < params->rx.n_queues; i++) { + status = rte_eth_rx_queue_setup( + port_id, + i, + params->rx.queue_size, + cpu_id, + NULL, + mempool->m); + + if (status < 0) + return NULL; + } + + /* Port TX */ + for (i = 0; i < params->tx.n_queues; i++) { + status = rte_eth_tx_queue_setup( + port_id, + i, + params->tx.queue_size, + cpu_id, + NULL); + + if (status < 0) + return NULL; + } + + /* Port start */ + status = rte_eth_dev_start(port_id); + if (status < 0) + return NULL; + + if (rss) { + status = rss_setup(port_id, port_info.reta_size, rss); + + if (status) { + rte_eth_dev_stop(port_id); + return NULL; + } + } + + /* Port link up */ + status = rte_eth_dev_set_link_up(port_id); + if ((status < 0) && (status != -ENOTSUP)) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node allocation */ + link = calloc(1, sizeof(struct link)); + if (link == NULL) { + rte_eth_dev_stop(port_id); + return NULL; + } + + /* Node fill in */ + strlcpy(link->name, name, sizeof(link->name)); + link->port_id = port_id; + rte_eth_dev_get_name_by_port(port_id, link->dev_name); + link->n_rxq = params->rx.n_queues; + link->n_txq = params->tx.n_queues; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->link_list, link, node); + + return link; +} + +int +link_is_up(struct obj *obj, const char *name) +{ + struct rte_eth_link link_params; + struct link *link; + + /* Check input params */ + if (!obj || !name) + return 0; + + link = link_find(obj, name); + if (link == NULL) + return 0; + + /* Resource */ + if (rte_eth_link_get(link->port_id, &link_params) < 0) + return 0; + + return (link_params.link_status == ETH_LINK_DOWN) ? 0 : 1; +} + +struct link * +link_find(struct obj *obj, const char *name) +{ + struct link *link; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(link, &obj->link_list, node) + if (strcmp(link->name, name) == 0) + return link; + + return NULL; +} + +struct link * +link_next(struct obj *obj, struct link *link) +{ + return (link == NULL) ? + TAILQ_FIRST(&obj->link_list) : TAILQ_NEXT(link, node); +} + +/* + * pipeline + */ +#ifndef PIPELINE_MSGQ_SIZE +#define PIPELINE_MSGQ_SIZE 64 +#endif + +struct pipeline * +pipeline_create(struct obj *obj, const char *name, int numa_node) +{ + struct pipeline *pipeline; + struct rte_swx_pipeline *p; + int status; + + /* Check input params */ + if ((name == NULL) || + pipeline_find(obj, name)) + return NULL; + + /* Resource create */ + status = rte_swx_pipeline_config(&p, numa_node); + if (status) + return NULL; + + status = rte_swx_pipeline_port_in_type_register(p, + "ethdev", + &rte_swx_port_ethdev_reader_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + + status = rte_swx_pipeline_port_out_type_register(p, + "ethdev", + &rte_swx_port_ethdev_writer_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + +#ifdef RTE_PORT_PCAP + + status = rte_swx_pipeline_port_in_type_register(p, + "source", + &rte_swx_port_source_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + + status = rte_swx_pipeline_port_out_type_register(p, + "sink", + &rte_swx_port_sink_ops); + if (status) { + rte_swx_pipeline_free(p); + return NULL; + } + +#endif + + /* Node allocation */ + pipeline = calloc(1, sizeof(struct pipeline)); + if (pipeline == NULL) { + rte_swx_pipeline_free(p); + return NULL; + } + + /* Node fill in */ + strlcpy(pipeline->name, name, sizeof(pipeline->name)); + pipeline->p = p; + pipeline->timer_period_ms = 10; + + /* Node add to list */ + TAILQ_INSERT_TAIL(&obj->pipeline_list, pipeline, node); + + return pipeline; +} + +struct pipeline * +pipeline_find(struct obj *obj, const char *name) +{ + struct pipeline *pipeline; + + if (!obj || !name) + return NULL; + + TAILQ_FOREACH(pipeline, &obj->pipeline_list, node) + if (strcmp(name, pipeline->name) == 0) + return pipeline; + + return NULL; +} + +/* + * obj + */ +struct obj * +obj_init(void) +{ + struct obj *obj; + + obj = calloc(1, sizeof(struct obj)); + if (!obj) + return NULL; + + TAILQ_INIT(&obj->mempool_list); + TAILQ_INIT(&obj->link_list); + TAILQ_INIT(&obj->pipeline_list); + + return obj; +} diff --git a/examples/pipeline/obj.h b/examples/pipeline/obj.h new file mode 100644 index 000000000..2f48b790f --- /dev/null +++ b/examples/pipeline/obj.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_OBJ_H_ +#define _INCLUDE_OBJ_H_ + +#include <stdint.h> +#include <sys/queue.h> + +#include <rte_mempool.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#ifndef NAME_SIZE +#define NAME_SIZE 64 +#endif + +/* + * obj + */ +struct obj; + +struct obj * +obj_init(void); + +/* + * mempool + */ +struct mempool_params { + uint32_t buffer_size; + uint32_t pool_size; + uint32_t cache_size; + uint32_t cpu_id; +}; + +struct mempool { + TAILQ_ENTRY(mempool) node; + char name[NAME_SIZE]; + struct rte_mempool *m; + uint32_t buffer_size; +}; + +struct mempool * +mempool_create(struct obj *obj, + const char *name, + struct mempool_params *params); + +struct mempool * +mempool_find(struct obj *obj, + const char *name); + +/* + * link + */ +#ifndef LINK_RXQ_RSS_MAX +#define LINK_RXQ_RSS_MAX 16 +#endif + +struct link_params_rss { + uint32_t queue_id[LINK_RXQ_RSS_MAX]; + uint32_t n_queues; +}; + +struct link_params { + const char *dev_name; + uint16_t port_id; /**< Valid only when *dev_name* is NULL. */ + + struct { + uint32_t n_queues; + uint32_t queue_size; + const char *mempool_name; + struct link_params_rss *rss; + } rx; + + struct { + uint32_t n_queues; + uint32_t queue_size; + } tx; + + int promiscuous; +}; + +struct link { + TAILQ_ENTRY(link) node; + char name[NAME_SIZE]; + char dev_name[NAME_SIZE]; + uint16_t port_id; + uint32_t n_rxq; + uint32_t n_txq; +}; + +struct link * +link_create(struct obj *obj, + const char *name, + struct link_params *params); + +int +link_is_up(struct obj *obj, const char *name); + +struct link * +link_find(struct obj *obj, const char *name); + +struct link * +link_next(struct obj *obj, struct link *link); + +/* + * pipeline + */ +struct pipeline { + TAILQ_ENTRY(pipeline) node; + char name[NAME_SIZE]; + + struct rte_swx_pipeline *p; + struct rte_swx_ctl_pipeline *ctl; + + uint32_t timer_period_ms; + int enabled; + uint32_t thread_id; + uint32_t cpu_id; +}; + +struct pipeline * +pipeline_create(struct obj *obj, + const char *name, + int numa_node); + +struct pipeline * +pipeline_find(struct obj *obj, const char *name); + +#endif /* _INCLUDE_OBJ_H_ */ diff --git a/examples/pipeline/thread.c b/examples/pipeline/thread.c new file mode 100644 index 000000000..0d1474c55 --- /dev/null +++ b/examples/pipeline/thread.c @@ -0,0 +1,549 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdlib.h> + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_lcore.h> +#include <rte_ring.h> + +#include <rte_table_acl.h> +#include <rte_table_array.h> +#include <rte_table_hash.h> +#include <rte_table_lpm.h> +#include <rte_table_lpm_ipv6.h> + +#include "obj.h" +#include "thread.h" + +#ifndef THREAD_PIPELINES_MAX +#define THREAD_PIPELINES_MAX 256 +#endif + +#ifndef THREAD_MSGQ_SIZE +#define THREAD_MSGQ_SIZE 64 +#endif + +#ifndef THREAD_TIMER_PERIOD_MS +#define THREAD_TIMER_PERIOD_MS 100 +#endif + +/** + * Master thead: data plane thread context + */ +struct thread { + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + + uint32_t enabled; +}; + +static struct thread thread[RTE_MAX_LCORE]; + +/** + * Data plane threads: context + */ +struct pipeline_data { + struct rte_swx_pipeline *p; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; +}; + +struct thread_data { + struct rte_swx_pipeline *p[THREAD_PIPELINES_MAX]; + uint32_t n_pipelines; + + struct pipeline_data pipeline_data[THREAD_PIPELINES_MAX]; + struct rte_ring *msgq_req; + struct rte_ring *msgq_rsp; + uint64_t timer_period; /* Measured in CPU cycles. */ + uint64_t time_next; + uint64_t time_next_min; +} __rte_cache_aligned; + +static struct thread_data thread_data[RTE_MAX_LCORE]; + +/** + * Master thread: data plane thread init + */ +static void +thread_free(void) +{ + uint32_t i; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct thread *t = &thread[i]; + + if (!rte_lcore_is_enabled(i)) + continue; + + /* MSGQs */ + if (t->msgq_req) + rte_ring_free(t->msgq_req); + + if (t->msgq_rsp) + rte_ring_free(t->msgq_rsp); + } +} + +int +thread_init(void) +{ + uint32_t i; + + RTE_LCORE_FOREACH_SLAVE(i) { + char name[NAME_MAX]; + struct rte_ring *msgq_req, *msgq_rsp; + struct thread *t = &thread[i]; + struct thread_data *t_data = &thread_data[i]; + uint32_t cpu_id = rte_lcore_to_socket_id(i); + + /* MSGQs */ + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-REQ", i); + + msgq_req = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_req == NULL) { + thread_free(); + return -1; + } + + snprintf(name, sizeof(name), "THREAD-%04x-MSGQ-RSP", i); + + msgq_rsp = rte_ring_create(name, + THREAD_MSGQ_SIZE, + cpu_id, + RING_F_SP_ENQ | RING_F_SC_DEQ); + + if (msgq_rsp == NULL) { + thread_free(); + return -1; + } + + /* Master thread records */ + t->msgq_req = msgq_req; + t->msgq_rsp = msgq_rsp; + t->enabled = 1; + + /* Data plane thread records */ + t_data->n_pipelines = 0; + t_data->msgq_req = msgq_req; + t_data->msgq_rsp = msgq_rsp; + t_data->timer_period = + (rte_get_tsc_hz() * THREAD_TIMER_PERIOD_MS) / 1000; + t_data->time_next = rte_get_tsc_cycles() + t_data->timer_period; + t_data->time_next_min = t_data->time_next; + } + + return 0; +} + +static inline int +thread_is_running(uint32_t thread_id) +{ + enum rte_lcore_state_t thread_state; + + thread_state = rte_eal_get_lcore_state(thread_id); + return (thread_state == RUNNING) ? 1 : 0; +} + +/** + * Master thread & data plane threads: message passing + */ +enum thread_req_type { + THREAD_REQ_PIPELINE_ENABLE = 0, + THREAD_REQ_PIPELINE_DISABLE, + THREAD_REQ_MAX +}; + +struct thread_msg_req { + enum thread_req_type type; + + union { + struct { + struct rte_swx_pipeline *p; + uint32_t timer_period_ms; + } pipeline_enable; + + struct { + struct rte_swx_pipeline *p; + } pipeline_disable; + }; +}; + +struct thread_msg_rsp { + int status; +}; + +/** + * Master thread + */ +static struct thread_msg_req * +thread_msg_alloc(void) +{ + size_t size = RTE_MAX(sizeof(struct thread_msg_req), + sizeof(struct thread_msg_rsp)); + + return calloc(1, size); +} + +static void +thread_msg_free(struct thread_msg_rsp *rsp) +{ + free(rsp); +} + +static struct thread_msg_rsp * +thread_msg_send_recv(uint32_t thread_id, + struct thread_msg_req *req) +{ + struct thread *t = &thread[thread_id]; + struct rte_ring *msgq_req = t->msgq_req; + struct rte_ring *msgq_rsp = t->msgq_rsp; + struct thread_msg_rsp *rsp; + int status; + + /* send */ + do { + status = rte_ring_sp_enqueue(msgq_req, req); + } while (status == -ENOBUFS); + + /* recv */ + do { + status = rte_ring_sc_dequeue(msgq_rsp, (void **) &rsp); + } while (status != 0); + + return rsp; +} + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + struct pipeline_data *tdp = &td->pipeline_data[td->n_pipelines]; + + if (td->n_pipelines >= THREAD_PIPELINES_MAX) + return -1; + + /* Data plane thread */ + td->p[td->n_pipelines] = p->p; + + tdp->p = p->p; + tdp->timer_period = + (rte_get_tsc_hz() * p->timer_period_ms) / 1000; + tdp->time_next = rte_get_tsc_cycles() + tdp->timer_period; + + td->n_pipelines++; + + /* Pipeline */ + p->thread_id = thread_id; + p->enabled = 1; + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_ENABLE; + req->pipeline_enable.p = p->p; + req->pipeline_enable.timer_period_ms = p->timer_period_ms; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->thread_id = thread_id; + p->enabled = 1; + + return 0; +} + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name) +{ + struct pipeline *p = pipeline_find(obj, pipeline_name); + struct thread *t; + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + int status; + + /* Check input params */ + if ((thread_id >= RTE_MAX_LCORE) || + (p == NULL)) + return -1; + + t = &thread[thread_id]; + if (t->enabled == 0) + return -1; + + if (p->enabled == 0) + return 0; + + if (p->thread_id != thread_id) + return -1; + + if (!thread_is_running(thread_id)) { + struct thread_data *td = &thread_data[thread_id]; + uint32_t i; + + for (i = 0; i < td->n_pipelines; i++) { + struct pipeline_data *tdp = &td->pipeline_data[i]; + + if (tdp->p != p->p) + continue; + + /* Data plane thread */ + if (i < td->n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + td->p[td->n_pipelines - 1]; + struct pipeline_data *tdp_last = + &td->pipeline_data[td->n_pipelines - 1]; + + td->p[i] = pipeline_last; + memcpy(tdp, tdp_last, sizeof(*tdp)); + } + + td->n_pipelines--; + + /* Pipeline */ + p->enabled = 0; + + break; + } + + return 0; + } + + /* Allocate request */ + req = thread_msg_alloc(); + if (req == NULL) + return -1; + + /* Write request */ + req->type = THREAD_REQ_PIPELINE_DISABLE; + req->pipeline_disable.p = p->p; + + /* Send request and wait for response */ + rsp = thread_msg_send_recv(thread_id, req); + + /* Read response */ + status = rsp->status; + + /* Free response */ + thread_msg_free(rsp); + + /* Request completion */ + if (status) + return status; + + p->enabled = 0; + + return 0; +} + +/** + * Data plane threads: message handling + */ +static inline struct thread_msg_req * +thread_msg_recv(struct rte_ring *msgq_req) +{ + struct thread_msg_req *req; + + int status = rte_ring_sc_dequeue(msgq_req, (void **) &req); + + if (status != 0) + return NULL; + + return req; +} + +static inline void +thread_msg_send(struct rte_ring *msgq_rsp, + struct thread_msg_rsp *rsp) +{ + int status; + + do { + status = rte_ring_sp_enqueue(msgq_rsp, rsp); + } while (status == -ENOBUFS); +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_enable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + struct pipeline_data *p = &t->pipeline_data[t->n_pipelines]; + + /* Request */ + if (t->n_pipelines >= THREAD_PIPELINES_MAX) { + rsp->status = -1; + return rsp; + } + + t->p[t->n_pipelines] = req->pipeline_enable.p; + + p->p = req->pipeline_enable.p; + p->timer_period = (rte_get_tsc_hz() * + req->pipeline_enable.timer_period_ms) / 1000; + p->time_next = rte_get_tsc_cycles() + p->timer_period; + + t->n_pipelines++; + + /* Response */ + rsp->status = 0; + return rsp; +} + +static struct thread_msg_rsp * +thread_msg_handle_pipeline_disable(struct thread_data *t, + struct thread_msg_req *req) +{ + struct thread_msg_rsp *rsp = (struct thread_msg_rsp *) req; + uint32_t n_pipelines = t->n_pipelines; + struct rte_swx_pipeline *pipeline = req->pipeline_disable.p; + uint32_t i; + + /* find pipeline */ + for (i = 0; i < n_pipelines; i++) { + struct pipeline_data *p = &t->pipeline_data[i]; + + if (p->p != pipeline) + continue; + + if (i < n_pipelines - 1) { + struct rte_swx_pipeline *pipeline_last = + t->p[n_pipelines - 1]; + struct pipeline_data *p_last = + &t->pipeline_data[n_pipelines - 1]; + + t->p[i] = pipeline_last; + memcpy(p, p_last, sizeof(*p)); + } + + t->n_pipelines--; + + rsp->status = 0; + return rsp; + } + + /* should not get here */ + rsp->status = 0; + return rsp; +} + +static void +thread_msg_handle(struct thread_data *t) +{ + for ( ; ; ) { + struct thread_msg_req *req; + struct thread_msg_rsp *rsp; + + req = thread_msg_recv(t->msgq_req); + if (req == NULL) + break; + + switch (req->type) { + case THREAD_REQ_PIPELINE_ENABLE: + rsp = thread_msg_handle_pipeline_enable(t, req); + break; + + case THREAD_REQ_PIPELINE_DISABLE: + rsp = thread_msg_handle_pipeline_disable(t, req); + break; + + default: + rsp = (struct thread_msg_rsp *) req; + rsp->status = -1; + } + + thread_msg_send(t->msgq_rsp, rsp); + } +} + +/** + * Data plane threads: main + */ +int +thread_main(void *arg __rte_unused) +{ + struct thread_data *t; + uint32_t thread_id, i; + + thread_id = rte_lcore_id(); + t = &thread_data[thread_id]; + + /* Dispatch loop */ + for (i = 0; ; i++) { + uint32_t j; + + /* Data Plane */ + for (j = 0; j < t->n_pipelines; j++) + rte_swx_pipeline_run(t->p[j], 1000000); + + /* Control Plane */ + if ((i & 0xF) == 0) { + uint64_t time = rte_get_tsc_cycles(); + uint64_t time_next_min = UINT64_MAX; + + if (time < t->time_next_min) + continue; + + /* Thread message queues */ + { + uint64_t time_next = t->time_next; + + if (time_next <= time) { + thread_msg_handle(t); + time_next = time + t->timer_period; + t->time_next = time_next; + } + + if (time_next < time_next_min) + time_next_min = time_next; + } + + t->time_next_min = time_next_min; + } + } + + return 0; +} diff --git a/examples/pipeline/thread.h b/examples/pipeline/thread.h new file mode 100644 index 000000000..829d82cbd --- /dev/null +++ b/examples/pipeline/thread.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef _INCLUDE_THREAD_H_ +#define _INCLUDE_THREAD_H_ + +#include <stdint.h> + +#include "obj.h" + +int +thread_pipeline_enable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_pipeline_disable(uint32_t thread_id, + struct obj *obj, + const char *pipeline_name); + +int +thread_init(void); + +int +thread_main(void *arg); + +#endif /* _INCLUDE_THREAD_H_ */ -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 36/40] examples/pipeline: add message passing mechanism 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (34 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 35/40] examples/pipeline: add new example application Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 37/40] examples/pipeline: add configuration commands Cristian Dumitrescu ` (3 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add network-based connectivity mechanism for the application to allow for the exchange of configuration messages through the network as opposed to local CLI only. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/conn.c | 331 ++++++++++++++++++++++++++++++++++ examples/pipeline/conn.h | 50 +++++ examples/pipeline/main.c | 136 +++++++++++++- examples/pipeline/meson.build | 1 + 5 files changed, 517 insertions(+), 2 deletions(-) create mode 100644 examples/pipeline/conn.c create mode 100644 examples/pipeline/conn.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 3c85e9e40..eb1085f7c 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c diff --git a/examples/pipeline/conn.c b/examples/pipeline/conn.c new file mode 100644 index 000000000..eed87b8ea --- /dev/null +++ b/examples/pipeline/conn.c @@ -0,0 +1,331 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <unistd.h> +#include <sys/types.h> + +#include <sys/socket.h> + +#include <sys/epoll.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <errno.h> + +#include "conn.h" + +#define MSG_CMD_TOO_LONG "Command too long." + +struct conn { + char *welcome; + char *prompt; + char *buf; + char *msg_in; + char *msg_out; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + size_t msg_in_len; + int fd_server; + int fd_client_group; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p) +{ + struct sockaddr_in server_address; + struct conn *conn; + int fd_server, fd_client_group, status; + + memset(&server_address, 0, sizeof(server_address)); + + /* Check input arguments */ + if ((p == NULL) || + (p->welcome == NULL) || + (p->prompt == NULL) || + (p->addr == NULL) || + (p->buf_size == 0) || + (p->msg_in_len_max == 0) || + (p->msg_out_len_max == 0) || + (p->msg_handle == NULL)) + return NULL; + + status = inet_aton(p->addr, &server_address.sin_addr); + if (status == 0) + return NULL; + + /* Memory allocation */ + conn = calloc(1, sizeof(struct conn)); + if (conn == NULL) + return NULL; + + conn->welcome = calloc(1, CONN_WELCOME_LEN_MAX + 1); + conn->prompt = calloc(1, CONN_PROMPT_LEN_MAX + 1); + conn->buf = calloc(1, p->buf_size); + conn->msg_in = calloc(1, p->msg_in_len_max + 1); + conn->msg_out = calloc(1, p->msg_out_len_max + 1); + + if ((conn->welcome == NULL) || + (conn->prompt == NULL) || + (conn->buf == NULL) || + (conn->msg_in == NULL) || + (conn->msg_out == NULL)) { + conn_free(conn); + return NULL; + } + + /* Server socket */ + server_address.sin_family = AF_INET; + server_address.sin_port = htons(p->port); + + fd_server = socket(AF_INET, + SOCK_STREAM | SOCK_NONBLOCK, + 0); + if (fd_server == -1) { + conn_free(conn); + return NULL; + } + + status = bind(fd_server, + (struct sockaddr *) &server_address, + sizeof(server_address)); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + status = listen(fd_server, 16); + if (status == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Client group */ + fd_client_group = epoll_create(1); + if (fd_client_group == -1) { + conn_free(conn); + close(fd_server); + return NULL; + } + + /* Fill in */ + strncpy(conn->welcome, p->welcome, CONN_WELCOME_LEN_MAX); + strncpy(conn->prompt, p->prompt, CONN_PROMPT_LEN_MAX); + conn->buf_size = p->buf_size; + conn->msg_in_len_max = p->msg_in_len_max; + conn->msg_out_len_max = p->msg_out_len_max; + conn->msg_in_len = 0; + conn->fd_server = fd_server; + conn->fd_client_group = fd_client_group; + conn->msg_handle = p->msg_handle; + conn->msg_handle_arg = p->msg_handle_arg; + + return conn; +} + +void +conn_free(struct conn *conn) +{ + if (conn == NULL) + return; + + if (conn->fd_client_group) + close(conn->fd_client_group); + + if (conn->fd_server) + close(conn->fd_server); + + free(conn->msg_out); + free(conn->msg_in); + free(conn->prompt); + free(conn->welcome); + free(conn); +} + +int +conn_poll_for_conn(struct conn *conn) +{ + struct sockaddr_in client_address; + struct epoll_event event; + socklen_t client_address_length; + int fd_client, status; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Server socket */ + client_address_length = sizeof(client_address); + fd_client = accept4(conn->fd_server, + (struct sockaddr *) &client_address, + &client_address_length, + SOCK_NONBLOCK); + if (fd_client == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + + /* Client group */ + event.events = EPOLLIN | EPOLLRDHUP | EPOLLHUP; + event.data.fd = fd_client; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_ADD, + fd_client, + &event); + if (status == -1) { + close(fd_client); + return -1; + } + + /* Client */ + status = write(fd_client, + conn->welcome, + strlen(conn->welcome)); + if (status == -1) { + close(fd_client); + return -1; + } + + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) { + close(fd_client); + return -1; + } + + return 0; +} + +static int +data_event_handle(struct conn *conn, + int fd_client) +{ + ssize_t len, i, status; + + /* Read input message */ + + len = read(fd_client, + conn->buf, + conn->buf_size); + if (len == -1) { + if ((errno == EAGAIN) || (errno == EWOULDBLOCK)) + return 0; + + return -1; + } + if (len == 0) + return 0; + + /* Handle input messages */ + for (i = 0; i < len; i++) { + if (conn->buf[i] == '\n') { + size_t n; + + conn->msg_in[conn->msg_in_len] = 0; + conn->msg_out[0] = 0; + + conn->msg_handle(conn->msg_in, + conn->msg_out, + conn->msg_out_len_max, + conn->msg_handle_arg); + + n = strlen(conn->msg_out); + if (n) { + status = write(fd_client, + conn->msg_out, + n); + if (status == -1) + return status; + } + + conn->msg_in_len = 0; + } else if (conn->msg_in_len < conn->msg_in_len_max) { + conn->msg_in[conn->msg_in_len] = conn->buf[i]; + conn->msg_in_len++; + } else { + status = write(fd_client, + MSG_CMD_TOO_LONG, + strlen(MSG_CMD_TOO_LONG)); + if (status == -1) + return status; + + conn->msg_in_len = 0; + } + } + + /* Write prompt */ + status = write(fd_client, + conn->prompt, + strlen(conn->prompt)); + if (status == -1) + return status; + + return 0; +} + +static int +control_event_handle(struct conn *conn, + int fd_client) +{ + int status; + + status = epoll_ctl(conn->fd_client_group, + EPOLL_CTL_DEL, + fd_client, + NULL); + if (status == -1) + return -1; + + status = close(fd_client); + if (status == -1) + return -1; + + return 0; +} + +int +conn_poll_for_msg(struct conn *conn) +{ + struct epoll_event event; + int fd_client, status, status_data = 0, status_control = 0; + + /* Check input arguments */ + if (conn == NULL) + return -1; + + /* Client group */ + status = epoll_wait(conn->fd_client_group, + &event, + 1, + 0); + if (status == -1) + return -1; + if (status == 0) + return 0; + + fd_client = event.data.fd; + + /* Data available */ + if (event.events & EPOLLIN) + status_data = data_event_handle(conn, fd_client); + + /* Control events */ + if (event.events & (EPOLLRDHUP | EPOLLERR | EPOLLHUP)) + status_control = control_event_handle(conn, fd_client); + + if (status_data || status_control) + return -1; + + return 0; +} diff --git a/examples/pipeline/conn.h b/examples/pipeline/conn.h new file mode 100644 index 000000000..871a5efd0 --- /dev/null +++ b/examples/pipeline/conn.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CONN_H__ +#define __INCLUDE_CONN_H__ + +#include <stdint.h> + +struct conn; + +#ifndef CONN_WELCOME_LEN_MAX +#define CONN_WELCOME_LEN_MAX 1024 +#endif + +#ifndef CONN_PROMPT_LEN_MAX +#define CONN_PROMPT_LEN_MAX 16 +#endif + +typedef void +(*conn_msg_handle_t)(char *msg_in, + char *msg_out, + size_t msg_out_len_max, + void *arg); + +struct conn_params { + const char *welcome; + const char *prompt; + const char *addr; + uint16_t port; + size_t buf_size; + size_t msg_in_len_max; + size_t msg_out_len_max; + conn_msg_handle_t msg_handle; + void *msg_handle_arg; +}; + +struct conn * +conn_init(struct conn_params *p); + +void +conn_free(struct conn *conn); + +int +conn_poll_for_conn(struct conn *conn); + +int +conn_poll_for_msg(struct conn *conn); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 353e62c10..1a63c1237 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,15 +11,136 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "conn.h" #include "obj.h" #include "thread.h" +static const char usage[] = + "%s EAL_ARGS -- [-h HOST] [-p PORT] [-s SCRIPT]\n"; + +static struct app_params { + struct conn_params conn; + char *script_name; +} app = { + .conn = { + .welcome = "\nWelcome!\n\n", + .prompt = "pipeline> ", + .addr = "0.0.0.0", + .port = 8086, + .buf_size = 1024 * 1024, + .msg_in_len_max = 1024, + .msg_out_len_max = 1024 * 1024, + .msg_handle = NULL, + .msg_handle_arg = NULL, /* set later. */ + }, + .script_name = NULL, +}; + +static int +parse_args(int argc, char **argv) +{ + char *app_name = argv[0]; + struct option lgopts[] = { + { NULL, 0, 0, 0 } + }; + int opt, option_index; + int h_present, p_present, s_present, n_args, i; + + /* Skip EAL input args */ + n_args = argc; + for (i = 0; i < n_args; i++) + if (strcmp(argv[i], "--") == 0) { + argc -= i; + argv += i; + break; + } + + if (i == n_args) + return 0; + + /* Parse args */ + h_present = 0; + p_present = 0; + s_present = 0; + + while ((opt = getopt_long(argc, argv, "h:p:s:", lgopts, &option_index)) + != EOF) + switch (opt) { + case 'h': + if (h_present) { + printf("Error: Multiple -h arguments\n"); + return -1; + } + h_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -h not provided\n"); + return -1; + } + + app.conn.addr = strdup(optarg); + if (app.conn.addr == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + case 'p': + if (p_present) { + printf("Error: Multiple -p arguments\n"); + return -1; + } + p_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -p not provided\n"); + return -1; + } + + app.conn.port = (uint16_t) atoi(optarg); + break; + + case 's': + if (s_present) { + printf("Error: Multiple -s arguments\n"); + return -1; + } + s_present = 1; + + if (!strlen(optarg)) { + printf("Error: Argument for -s not provided\n"); + return -1; + } + + app.script_name = strdup(optarg); + if (app.script_name == NULL) { + printf("Error: Not enough memory\n"); + return -1; + } + break; + + default: + printf(usage, app_name); + return -1; + } + + optind = 1; /* reset getopt lib */ + + return 0; +} + int main(int argc, char **argv) { + struct conn *conn; struct obj *obj; int status; + /* Parse application arguments */ + status = parse_args(argc, argv); + if (status < 0) + return status; + /* EAL */ status = rte_eal_init(argc, argv); if (status < 0) { @@ -46,7 +167,18 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Connectivity */ + app.conn.msg_handle_arg = obj; + conn = conn_init(&app.conn); + if (!conn) { + printf("Error: Connectivity initialization failed (%d)\n", + status); + return status; + }; /* Dispatch loop */ - for ( ; ; ); -} + for ( ; ; ) { + conn_poll_for_conn(conn); + conn_poll_for_msg(conn); + } +} diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index ade485f97..a92e84677 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'conn.c', 'main.c', 'obj.c', 'thread.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 37/40] examples/pipeline: add configuration commands 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (35 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 36/40] examples/pipeline: add message passing mechanism Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 38/40] examples/pipeline: add l2fwd example Cristian Dumitrescu ` (2 subsequent siblings) 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add CLI commands for application configuration and query. Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 1373 +++++++++++++++++++++++++++++++++ examples/pipeline/cli.h | 19 + examples/pipeline/main.c | 11 +- examples/pipeline/meson.build | 1 + 5 files changed, 1404 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/cli.c create mode 100644 examples/pipeline/cli.h diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index eb1085f7c..dcc0f67bf 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -5,6 +5,7 @@ APP = pipeline # all source are stored in SRCS-y +SRCS-y += cli.c SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c new file mode 100644 index 000000000..f68a1a924 --- /dev/null +++ b/examples/pipeline/cli.c @@ -0,0 +1,1373 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +#include <rte_common.h> +#include <rte_ethdev.h> +#include <rte_swx_port_ethdev.h> +#include <rte_swx_port_source_sink.h> +#include <rte_swx_pipeline.h> +#include <rte_swx_ctl.h> + +#include "cli.h" + +#include "obj.h" +#include "thread.h" + +#ifndef CMD_MAX_TOKENS +#define CMD_MAX_TOKENS 256 +#endif + +#define MSG_OUT_OF_MEMORY "Not enough memory.\n" +#define MSG_CMD_UNKNOWN "Unknown command \"%s\".\n" +#define MSG_CMD_UNIMPLEM "Command \"%s\" not implemented.\n" +#define MSG_ARG_NOT_ENOUGH "Not enough arguments for command \"%s\".\n" +#define MSG_ARG_TOO_MANY "Too many arguments for command \"%s\".\n" +#define MSG_ARG_MISMATCH "Wrong number of arguments for command \"%s\".\n" +#define MSG_ARG_NOT_FOUND "Argument \"%s\" not found.\n" +#define MSG_ARG_INVALID "Invalid value for argument \"%s\".\n" +#define MSG_FILE_ERR "Error in file \"%s\" at line %u.\n" +#define MSG_FILE_NOT_ENOUGH "Not enough rules in file \"%s\".\n" +#define MSG_CMD_FAIL "Command \"%s\" failed.\n" + +#define skip_white_spaces(pos) \ +({ \ + __typeof__(pos) _p = (pos); \ + for ( ; isspace(*_p); _p++) \ + ; \ + _p; \ +}) + +static int +parser_read_uint64(uint64_t *value, const char *p) +{ + char *next; + uint64_t val; + + p = skip_white_spaces(p); + if (!isdigit(*p)) + return -EINVAL; + + val = strtoul(p, &next, 10); + if (p == next) + return -EINVAL; + + p = next; + switch (*p) { + case 'T': + val *= 1024ULL; + /* fall through */ + case 'G': + val *= 1024ULL; + /* fall through */ + case 'M': + val *= 1024ULL; + /* fall through */ + case 'k': + case 'K': + val *= 1024ULL; + p++; + break; + } + + p = skip_white_spaces(p); + if (*p != '\0') + return -EINVAL; + + *value = val; + return 0; +} + +static int +parser_read_uint32(uint32_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT32_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +static int +parser_read_uint16(uint16_t *value, const char *p) +{ + uint64_t val = 0; + int ret = parser_read_uint64(&val, p); + + if (ret < 0) + return ret; + + if (val > UINT16_MAX) + return -ERANGE; + + *value = val; + return 0; +} + +#define PARSE_DELIMITER " \f\n\r\t\v" + +static int +parse_tokenize_string(char *string, char *tokens[], uint32_t *n_tokens) +{ + uint32_t i; + + if ((string == NULL) || + (tokens == NULL) || + (*n_tokens < 1)) + return -EINVAL; + + for (i = 0; i < *n_tokens; i++) { + tokens[i] = strtok_r(string, PARSE_DELIMITER, &string); + if (tokens[i] == NULL) + break; + } + + if ((i == *n_tokens) && strtok_r(string, PARSE_DELIMITER, &string)) + return -E2BIG; + + *n_tokens = i; + return 0; +} + +static int +is_comment(char *in) +{ + if ((strlen(in) && index("!#%;", in[0])) || + (strncmp(in, "//", 2) == 0) || + (strncmp(in, "--", 2) == 0)) + return 1; + + return 0; +} + +static const char cmd_mempool_help[] = +"mempool <mempool_name>\n" +" buffer <buffer_size>\n" +" pool <pool_size>\n" +" cache <cache_size>\n" +" cpu <cpu_id>\n"; + +static void +cmd_mempool(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct mempool_params p; + char *name; + struct mempool *mempool; + + if (n_tokens != 10) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (strcmp(tokens[2], "buffer") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "buffer"); + return; + } + + if (parser_read_uint32(&p.buffer_size, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "buffer_size"); + return; + } + + if (strcmp(tokens[4], "pool") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pool"); + return; + } + + if (parser_read_uint32(&p.pool_size, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "pool_size"); + return; + } + + if (strcmp(tokens[6], "cache") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cache"); + return; + } + + if (parser_read_uint32(&p.cache_size, tokens[7]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cache_size"); + return; + } + + if (strcmp(tokens[8], "cpu") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "cpu"); + return; + } + + if (parser_read_uint32(&p.cpu_id, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "cpu_id"); + return; + } + + mempool = mempool_create(obj, name, &p); + if (mempool == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +static const char cmd_link_help[] = +"link <link_name>\n" +" dev <device_name> | port <port_id>\n" +" rxq <n_queues> <queue_size> <mempool_name>\n" +" txq <n_queues> <queue_size>\n" +" promiscuous on | off\n" +" [rss <qid_0> ... <qid_n>]\n"; + +static void +cmd_link(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link_params p; + struct link_params_rss rss; + struct link *link; + char *name; + + memset(&p, 0, sizeof(p)); + + if ((n_tokens < 13) || (n_tokens > 14 + LINK_RXQ_RSS_MAX)) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + name = tokens[1]; + + if (strcmp(tokens[2], "dev") == 0) + p.dev_name = tokens[3]; + else if (strcmp(tokens[2], "port") == 0) { + p.dev_name = NULL; + + if (parser_read_uint16(&p.port_id, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + } else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "dev or port"); + return; + } + + if (strcmp(tokens[4], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint32(&p.rx.n_queues, tokens[5]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + if (parser_read_uint32(&p.rx.queue_size, tokens[6]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + p.rx.mempool_name = tokens[7]; + + if (strcmp(tokens[8], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint32(&p.tx.n_queues, tokens[9]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "n_queues"); + return; + } + + if (parser_read_uint32(&p.tx.queue_size, tokens[10]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "queue_size"); + return; + } + + if (strcmp(tokens[11], "promiscuous") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "promiscuous"); + return; + } + + if (strcmp(tokens[12], "on") == 0) + p.promiscuous = 1; + else if (strcmp(tokens[12], "off") == 0) + p.promiscuous = 0; + else { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "on or off"); + return; + } + + /* RSS */ + p.rx.rss = NULL; + if (n_tokens > 13) { + uint32_t queue_id, i; + + if (strcmp(tokens[13], "rss") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rss"); + return; + } + + p.rx.rss = &rss; + + rss.n_queues = 0; + for (i = 14; i < n_tokens; i++) { + if (parser_read_uint32(&queue_id, tokens[i]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + rss.queue_id[rss.n_queues] = queue_id; + rss.n_queues++; + } + } + + link = link_create(obj, name, &p); + if (link == NULL) { + snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); + return; + } +} + +/* Print the link stats and info */ +static void +print_link_info(struct link *link, char *out, size_t out_size) +{ + struct rte_eth_stats stats; + struct rte_ether_addr mac_addr; + struct rte_eth_link eth_link; + uint16_t mtu; + int ret; + + memset(&stats, 0, sizeof(stats)); + rte_eth_stats_get(link->port_id, &stats); + + ret = rte_eth_macaddr_get(link->port_id, &mac_addr); + if (ret != 0) { + snprintf(out, out_size, "\n%s: MAC address get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + ret = rte_eth_link_get(link->port_id, ð_link); + if (ret < 0) { + snprintf(out, out_size, "\n%s: link get failed: %s", + link->name, rte_strerror(-ret)); + return; + } + + rte_eth_dev_get_mtu(link->port_id, &mtu); + + snprintf(out, out_size, + "\n" + "%s: flags=<%s> mtu %u\n" + "\tether %02X:%02X:%02X:%02X:%02X:%02X rxqueues %u txqueues %u\n" + "\tport# %u speed %u Mbps\n" + "\tRX packets %" PRIu64" bytes %" PRIu64"\n" + "\tRX errors %" PRIu64" missed %" PRIu64" no-mbuf %" PRIu64"\n" + "\tTX packets %" PRIu64" bytes %" PRIu64"\n" + "\tTX errors %" PRIu64"\n", + link->name, + eth_link.link_status == 0 ? "DOWN" : "UP", + mtu, + mac_addr.addr_bytes[0], mac_addr.addr_bytes[1], + mac_addr.addr_bytes[2], mac_addr.addr_bytes[3], + mac_addr.addr_bytes[4], mac_addr.addr_bytes[5], + link->n_rxq, + link->n_txq, + link->port_id, + eth_link.link_speed, + stats.ipackets, + stats.ibytes, + stats.ierrors, + stats.imissed, + stats.rx_nombuf, + stats.opackets, + stats.obytes, + stats.oerrors); +} + +/* + * link show [<link_name>] + */ +static void +cmd_link_show(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct link *link; + char *link_name; + + if (n_tokens != 2 && n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (n_tokens == 2) { + link = link_next(obj, NULL); + + while (link != NULL) { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + print_link_info(link, out, out_size); + link = link_next(obj, link); + } + } else { + out_size = out_size - strlen(out); + out = &out[strlen(out)]; + + link_name = tokens[2]; + link = link_find(obj, link_name); + + if (link == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "Link does not exist"); + return; + } + print_link_info(link, out, out_size); + } +} + +static const char cmd_pipeline_create_help[] = +"pipeline <pipeline_name> create <numa_node>\n"; + +static void +cmd_pipeline_create(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + uint32_t numa_node; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + + if (parser_read_uint32(&numa_node, tokens[3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "numa_node"); + return; + } + + p = pipeline_create(obj, name, (int)numa_node); + if (!p) { + snprintf(out, out_size, "pipeline create error."); + return; + } +} + +static const char cmd_pipeline_port_in_help[] = +"pipeline <pipeline_name> port in <port_id>\n" +" link <link_name> rxq <queue_id> bsz <burst_size>\n" +" source <mempool_name> <fie_name>\n"; + +static void +cmd_pipeline_port_in(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "in") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "in"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_reader_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "rxq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "rxq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "source") == 0) { + struct rte_swx_port_source_params params; + struct mempool *mp; + + if (n_tokens < t0 + 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port in source"); + return; + } + + mp = mempool_find(obj, tokens[t0 + 1]); + if (!mp) { + snprintf(out, out_size, MSG_ARG_INVALID, + "mempool_name"); + return; + } + params.pool = mp->m; + + params.file_name = tokens[t0 + 2]; + + t0 += 3; + + status = rte_swx_pipeline_port_in_config(p->p, + port_id, + "source", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port in error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_port_out_help[] = +"pipeline <pipeline_name> port out <port_id>\n" +" link <link_name> txq <txq_id> bsz <burst_size>\n" +" | sink <file_name> | none\n"; + +static void +cmd_pipeline_port_out(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + int status; + uint32_t port_id = 0, t0; + + if (n_tokens < 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (strcmp(tokens[2], "port") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "port"); + return; + } + + if (strcmp(tokens[3], "out") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "out"); + return; + } + + if (parser_read_uint32(&port_id, tokens[4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "port_id"); + return; + } + + t0 = 5; + + if (strcmp(tokens[t0], "link") == 0) { + struct rte_swx_port_ethdev_writer_params params; + struct link *link; + + if (n_tokens < t0 + 6) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "pipeline port out link"); + return; + } + + link = link_find(obj, tokens[t0 + 1]); + if (!link) { + snprintf(out, out_size, MSG_ARG_INVALID, + "link_name"); + return; + } + params.dev_name = link->dev_name; + + if (strcmp(tokens[t0 + 2], "txq") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "txq"); + return; + } + + if (parser_read_uint16(¶ms.queue_id, tokens[t0 + 3]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_id"); + return; + } + + if (strcmp(tokens[t0 + 4], "bsz") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "bsz"); + return; + } + + if (parser_read_uint32(¶ms.burst_size, tokens[t0 + 5])) { + snprintf(out, out_size, MSG_ARG_INVALID, + "burst_size"); + return; + } + + t0 += 6; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "ethdev", + ¶ms); + } else if (strcmp(tokens[t0], "sink") == 0) { + struct rte_swx_port_sink_params params; + + params.file_name = strcmp(tokens[t0 + 1], "none") ? + tokens[t0 + 1] : NULL; + + t0 += 2; + + status = rte_swx_pipeline_port_out_config(p->p, + port_id, + "sink", + ¶ms); + } else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "port out error."); + return; + } + + if (n_tokens != t0) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } +} + +static const char cmd_pipeline_build_help[] = +"pipeline <pipeline_name> build <app>\n"; + +static void +cmd_pipeline_build(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *name; + int status; + + if (n_tokens != 4) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + name = tokens[1]; + p = pipeline_find(obj, name); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + status = rte_swx_pipeline_build(p->p); + if (status) { + snprintf(out, out_size, "Pipeline build failed."); + return; + } + + p->ctl = rte_swx_ctl_pipeline_create(p->p); + if (!p->ctl) { + snprintf(out, out_size, "Pipeline control create failed."); + rte_swx_pipeline_free(p->p); + return; + } +} + +static const char cmd_pipeline_table_update_help[] = +"pipeline <pipeline_name> table <table_name> update <file_name_add> " +"<file_name_delete> <file_name_default>"; + +static void +cmd_pipeline_table_update(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct pipeline *p; + char *pipeline_name, *table_name, *line = NULL; + char *file_name_add, *file_name_delete, *file_name_default; + FILE *file_add = NULL, *file_delete = NULL, *file_default = NULL; + uint32_t line_id; + int status; + + if (n_tokens != 8) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + pipeline_name = tokens[1]; + p = pipeline_find(obj, pipeline_name); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "table") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "table"); + return; + } + + table_name = tokens[3]; + + if (strcmp(tokens[4], "update") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "update"); + return; + } + + file_name_add = tokens[5]; + file_name_delete = tokens[6]; + file_name_default = tokens[7]; + + /* File open. */ + if (strcmp(file_name_add, "none")) { + file_add = fopen(file_name_add, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_add); + goto error; + } + } + + if (strcmp(file_name_delete, "none")) { + file_add = fopen(file_name_delete, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_delete); + goto error; + } + } + + if (strcmp(file_name_default, "none")) { + file_add = fopen(file_name_default, "r"); + if (!file_add) { + snprintf(out, out_size, "Cannot open file %s", + file_name_default); + goto error; + } + } + + if (!file_add && !file_delete && !file_default) { + snprintf(out, out_size, "Nothing to be done."); + return; + } + + /* Buffer allocation. */ + line = malloc(2048); + if (!line) { + snprintf(out, out_size, MSG_OUT_OF_MEMORY); + goto error; + } + + /* Add. */ + if (file_add) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_add) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_add, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_add, line_id); + goto error; + } + } + + fclose(file_add); + } + + /* Delete. */ + if (file_delete) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_delete) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_delete, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_entry_delete(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_delete, line_id); + goto error; + } + } + + fclose(file_delete); + } + + /* Default. */ + if (file_default) { + for (line_id = 1; ; line_id++) { + struct rte_swx_table_entry *entry; + + if (fgets(line, 2048, file_default) == NULL) + break; + + entry = rte_swx_ctl_pipeline_table_entry_read(p->ctl, + table_name, + line); + if (!entry) { + snprintf(out, out_size, MSG_FILE_ERR, + file_name_default, line_id); + goto error; + } + + status = rte_swx_ctl_pipeline_table_default_entry_add(p->ctl, + table_name, + entry); + if (status) { + snprintf(out, out_size, + "Invalid entry in file %s at line %u", + file_name_default, line_id); + goto error; + } + } + + fclose(file_default); + } + + status = rte_swx_ctl_pipeline_commit(p->ctl, 1); + if (status) { + snprintf(out, out_size, "Commit failed."); + goto error; + } + + free(line); + + rte_swx_ctl_pipeline_table_fprintf(stdout, p->ctl, table_name); + + return; + +error: + rte_swx_ctl_pipeline_abort(p->ctl); + free(line); + if (file_add) + fclose(file_add); + if (file_delete) + fclose(file_delete); + if (file_default) + fclose(file_default); +} + +static const char cmd_pipeline_stats_help[] = +"pipeline <pipeline_name> stats\n"; + +static void +cmd_pipeline_stats(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + struct rte_swx_ctl_pipeline_info info; + struct pipeline *p; + uint32_t i; + int status; + + if (n_tokens != 3) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + p = pipeline_find(obj, tokens[1]); + if (!p) { + snprintf(out, out_size, MSG_ARG_INVALID, "pipeline_name"); + return; + } + + if (strcmp(tokens[2], "stats")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "stats"); + return; + } + + status = rte_swx_ctl_pipeline_info_get(p->p, &info); + if (status) { + snprintf(out, out_size, "Pipeline info get error."); + return; + } + + snprintf(out, out_size, "Input ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_in; i++) { + struct rte_swx_port_in_stats stats; + + rte_swx_ctl_pipeline_port_in_stats_read(p->p, i, &stats); + + snprintf(out, out_size, + "\tPort %u: packets %lu bytes %lu empty %lu\n", + i, stats.n_pkts, stats.n_bytes, stats.n_empty); + out_size -= strlen(out); + out += strlen(out); + } + + snprintf(out, out_size, "Output ports:\n"); + out_size -= strlen(out); + out += strlen(out); + + for (i = 0; i < info.n_ports_out; i++) { + struct rte_swx_port_out_stats stats; + + rte_swx_ctl_pipeline_port_out_stats_read(p->p, i, &stats); + + snprintf(out, out_size, + "\tPort %u: packets %lu bytes %lu\n", + i, stats.n_pkts, stats.n_bytes); + out_size -= strlen(out); + out += strlen(out); + } +} + +static const char cmd_thread_pipeline_enable_help[] = +"thread <thread_id> pipeline <pipeline_name> enable\n"; + +static void +cmd_thread_pipeline_enable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + + if (strcmp(tokens[4], "enable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "enable"); + return; + } + + status = thread_pipeline_enable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, "thread pipeline enable"); + return; + } +} + +static const char cmd_thread_pipeline_disable_help[] = +"thread <thread_id> pipeline <pipeline_name> disable\n"; + +static void +cmd_thread_pipeline_disable(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *obj) +{ + char *pipeline_name; + uint32_t thread_id; + int status; + + if (n_tokens != 5) { + snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); + return; + } + + if (parser_read_uint32(&thread_id, tokens[1]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, "thread_id"); + return; + } + + if (strcmp(tokens[2], "pipeline") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "pipeline"); + return; + } + + pipeline_name = tokens[3]; + + if (strcmp(tokens[4], "disable") != 0) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, "disable"); + return; + } + + status = thread_pipeline_disable(thread_id, obj, pipeline_name); + if (status) { + snprintf(out, out_size, MSG_CMD_FAIL, + "thread pipeline disable"); + return; + } +} + +static void +cmd_help(char **tokens, + uint32_t n_tokens, + char *out, + size_t out_size, + void *arg __rte_unused) +{ + tokens++; + n_tokens--; + + if (n_tokens == 0) { + snprintf(out, out_size, + "Type 'help <command>' for command details.\n\n"); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_mempool_help); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + snprintf(out, out_size, "\n%s\n", cmd_link_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens == 1) && (strcmp(tokens[2], "create")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_create_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + (strcmp(tokens[1], "port") == 0)) { + if ((n_tokens == 3) && (strcmp(tokens[2], "in")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_in_help); + return; + } + + if ((n_tokens == 3) && (strcmp(tokens[2], "out")) == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_port_out_help); + return; + } + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "build")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_build_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "table")) == 0)) { + snprintf(out, out_size, "\n%s\n", + cmd_pipeline_table_update_help); + return; + } + + if ((strcmp(tokens[0], "pipeline") == 0) && + ((n_tokens >= 2) && (strcmp(tokens[2], "stats")) == 0)) { + snprintf(out, out_size, "\n%s\n", cmd_pipeline_stats_help); + return; + } + + if ((n_tokens == 3) && + (strcmp(tokens[0], "thread") == 0) && + (strcmp(tokens[1], "pipeline") == 0)) { + if (strcmp(tokens[2], "enable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_enable_help); + return; + } + + if (strcmp(tokens[2], "disable") == 0) { + snprintf(out, out_size, "\n%s\n", + cmd_thread_pipeline_disable_help); + return; + } + } + + snprintf(out, out_size, "Invalid command\n"); +} + +void +cli_process(char *in, char *out, size_t out_size, void *obj) +{ + char *tokens[CMD_MAX_TOKENS]; + uint32_t n_tokens = RTE_DIM(tokens); + int status; + + if (is_comment(in)) + return; + + status = parse_tokenize_string(in, tokens, &n_tokens); + if (status) { + snprintf(out, out_size, MSG_ARG_TOO_MANY, ""); + return; + } + + if (n_tokens == 0) + return; + + if (strcmp(tokens[0], "help") == 0) { + cmd_help(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "mempool") == 0) { + cmd_mempool(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "link") == 0) { + if (strcmp(tokens[1], "show") == 0) { + cmd_link_show(tokens, n_tokens, out, out_size, obj); + return; + } + + cmd_link(tokens, n_tokens, out, out_size, obj); + return; + } + + if (strcmp(tokens[0], "pipeline") == 0) { + if ((n_tokens >= 3) && + (strcmp(tokens[2], "create") == 0)) { + cmd_pipeline_create(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "in") == 0)) { + cmd_pipeline_port_in(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 4) && + (strcmp(tokens[2], "port") == 0) && + (strcmp(tokens[3], "out") == 0)) { + cmd_pipeline_port_out(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "build") == 0)) { + cmd_pipeline_build(tokens, n_tokens, out, out_size, + obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "table") == 0)) { + cmd_pipeline_table_update(tokens, n_tokens, out, + out_size, obj); + return; + } + + if ((n_tokens >= 3) && + (strcmp(tokens[2], "stats") == 0)) { + cmd_pipeline_stats(tokens, n_tokens, out, out_size, + obj); + return; + } + } + + if (strcmp(tokens[0], "thread") == 0) { + if ((n_tokens >= 5) && + (strcmp(tokens[4], "enable") == 0)) { + cmd_thread_pipeline_enable(tokens, n_tokens, + out, out_size, obj); + return; + } + + if ((n_tokens >= 5) && + (strcmp(tokens[4], "disable") == 0)) { + cmd_thread_pipeline_disable(tokens, n_tokens, + out, out_size, obj); + return; + } + } + + snprintf(out, out_size, MSG_CMD_UNKNOWN, tokens[0]); +} + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *obj) +{ + char *msg_in = NULL, *msg_out = NULL; + FILE *f = NULL; + + /* Check input arguments */ + if ((file_name == NULL) || + (strlen(file_name) == 0) || + (msg_in_len_max == 0) || + (msg_out_len_max == 0)) + return -EINVAL; + + msg_in = malloc(msg_in_len_max + 1); + msg_out = malloc(msg_out_len_max + 1); + if ((msg_in == NULL) || + (msg_out == NULL)) { + free(msg_out); + free(msg_in); + return -ENOMEM; + } + + /* Open input file */ + f = fopen(file_name, "r"); + if (f == NULL) { + free(msg_out); + free(msg_in); + return -EIO; + } + + /* Read file */ + for ( ; ; ) { + if (fgets(msg_in, msg_in_len_max + 1, f) == NULL) + break; + + printf("%s", msg_in); + msg_out[0] = 0; + + cli_process(msg_in, + msg_out, + msg_out_len_max, + obj); + + if (strlen(msg_out)) + printf("%s", msg_out); + } + + /* Close file */ + fclose(f); + free(msg_out); + free(msg_in); + return 0; +} diff --git a/examples/pipeline/cli.h b/examples/pipeline/cli.h new file mode 100644 index 000000000..dad7233fe --- /dev/null +++ b/examples/pipeline/cli.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2010-2020 Intel Corporation + */ + +#ifndef __INCLUDE_CLI_H__ +#define __INCLUDE_CLI_H__ + +#include <stddef.h> + +void +cli_process(char *in, char *out, size_t out_size, void *arg); + +int +cli_script_process(const char *file_name, + size_t msg_in_len_max, + size_t msg_out_len_max, + void *arg); + +#endif diff --git a/examples/pipeline/main.c b/examples/pipeline/main.c index 1a63c1237..97bb66288 100644 --- a/examples/pipeline/main.c +++ b/examples/pipeline/main.c @@ -11,6 +11,7 @@ #include <rte_launch.h> #include <rte_eal.h> +#include "cli.h" #include "conn.h" #include "obj.h" #include "thread.h" @@ -30,7 +31,7 @@ static struct app_params { .buf_size = 1024 * 1024, .msg_in_len_max = 1024, .msg_out_len_max = 1024 * 1024, - .msg_handle = NULL, + .msg_handle = cli_process, .msg_handle_arg = NULL, /* set later. */ }, .script_name = NULL, @@ -167,6 +168,13 @@ main(int argc, char **argv) NULL, SKIP_MASTER); + /* Script */ + if (app.script_name) + cli_script_process(app.script_name, + app.conn.msg_in_len_max, + app.conn.msg_out_len_max, + obj); + /* Connectivity */ app.conn.msg_handle_arg = obj; conn = conn_init(&app.conn); @@ -175,6 +183,7 @@ main(int argc, char **argv) status); return status; }; + /* Dispatch loop */ for ( ; ; ) { conn_poll_for_conn(conn); diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index a92e84677..4f47dec3e 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -10,6 +10,7 @@ build = cc.has_header('sys/epoll.h') deps += ['pipeline', 'bus_pci'] allow_experimental_apis = true sources = files( + 'cli.c', 'conn.c', 'main.c', 'obj.c', -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 38/40] examples/pipeline: add l2fwd example 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (36 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 37/40] examples/pipeline: add configuration commands Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 39/40] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example Cristian Dumitrescu 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add L2 Forwarding example to the pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 1 + examples/pipeline/cli.c | 16 ++- examples/pipeline/example_l2fwd.c | 125 ++++++++++++++++++++++ examples/pipeline/examples/l2fwd.cli | 25 +++++ examples/pipeline/examples/l2fwd_pcap.cli | 20 ++++ examples/pipeline/examples/packet.txt | 102 ++++++++++++++++++ examples/pipeline/meson.build | 1 + 7 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/example_l2fwd.c create mode 100644 examples/pipeline/examples/l2fwd.cli create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli create mode 100644 examples/pipeline/examples/packet.txt diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index dcc0f67bf..405978ced 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -10,6 +10,7 @@ SRCS-y += conn.c SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c +SRCS-y += example_l2fwd.c # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index f68a1a924..a54cd79ff 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -730,6 +730,7 @@ cmd_pipeline_port_out(char **tokens, static const char cmd_pipeline_build_help[] = "pipeline <pipeline_name> build <app>\n"; +int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); static void cmd_pipeline_build(char **tokens, uint32_t n_tokens, @@ -738,7 +739,7 @@ cmd_pipeline_build(char **tokens, void *obj) { struct pipeline *p; - char *name; + char *name, *app; int status; if (n_tokens != 4) { @@ -753,6 +754,19 @@ cmd_pipeline_build(char **tokens, return; } + app = tokens[3]; + if (!strcmp(app, "l2fwd")) + status = pipeline_setup_l2fwd(p->p); + else { + snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); + return; + } + + if (status) { + snprintf(out, out_size, "Pipeline build: app setup error."); + return; + } + status = rte_swx_pipeline_build(p->p); if (status) { snprintf(out, out_size, "Pipeline build failed."); diff --git a/examples/pipeline/example_l2fwd.c b/examples/pipeline/example_l2fwd.c new file mode 100644 index 000000000..a0b627709 --- /dev/null +++ b/examples/pipeline/example_l2fwd.c @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_port_ethdev.h" +#include "rte_swx_port_source_sink.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_type[] = { + {"port_in", 32}, + {"port_out", 32}, +}; + +/* + * Actions. + */ +static const char *action_passthrough_instructions[] = { + "return", +}; + +/* + * Tables. + */ +static const char *table_stub_actions[] = {"passthrough"}; + +static struct rte_swx_pipeline_table_params table_stub_params = { + /* Match. */ + .fields = NULL, + .n_fields = 0, + + /* Action. */ + .action_names = table_stub_actions, + .n_actions = RTE_DIM(table_stub_actions), + .default_action_name = "passthrough", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port_in", + "table stub", + "tx m.port_in", +}; + +int +pipeline_setup_l2fwd(struct rte_swx_pipeline *p); + +int +pipeline_setup_l2fwd(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_type", + metadata_type, + RTE_DIM(metadata_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_type"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "passthrough", + NULL, + action_passthrough_instructions, + RTE_DIM(action_passthrough_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_config(p, + "stub", + &table_stub_params, + NULL, + NULL, + 0); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/l2fwd.cli b/examples/pipeline/examples/l2fwd.cli new file mode 100644 index 000000000..15f8b1782 --- /dev/null +++ b/examples/pipeline/examples/l2fwd.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build l2fwd + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_pcap.cli b/examples/pipeline/examples/l2fwd_pcap.cli new file mode 100644 index 000000000..f9c37ab7e --- /dev/null +++ b/examples/pipeline/examples/l2fwd_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build l2fwd + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/packet.txt b/examples/pipeline/examples/packet.txt new file mode 100644 index 000000000..d1c79b7e7 --- /dev/null +++ b/examples/pipeline/examples/packet.txt @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +#Text to PCAP: text2pcap packet.txt packet.pcap +#PCAP to text: tcpdump -r packet.pcap -xx + +#Packet 0 +000000 aa bb cc dd 00 00 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 1 +000000 aa bb cc dd 00 01 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 2 +000000 aa bb cc dd 00 02 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 3 +000000 aa bb cc dd 00 03 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 4 +000000 aa bb cc dd 00 04 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 5 +000000 aa bb cc dd 00 05 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 6 +000000 aa bb cc dd 00 06 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 7 +000000 aa bb cc dd 00 07 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 8 +000000 aa bb cc dd 00 08 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 9 +000000 aa bb cc dd 00 09 b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 10 +000000 aa bb cc dd 00 0a b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 11 +000000 aa bb cc dd 00 0b b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 12 +000000 aa bb cc dd 00 0c b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 13 +000000 aa bb cc dd 00 0d b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 14 +000000 aa bb cc dd 00 0e b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 + +#Packet 15 +000000 aa bb cc dd 00 0f b0 b1 b2 b3 b4 b5 08 00 45 00 +000010 00 2e 00 00 00 00 40 11 00 00 c0 c1 c2 c3 d0 d1 +000020 d2 d3 e0 e1 f0 f1 00 1a 00 00 00 01 02 03 04 05 +000030 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index 4f47dec3e..b16de3714 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -15,4 +15,5 @@ sources = files( 'main.c', 'obj.c', 'thread.c', + 'example_l2fwd.c', ) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 39/40] examples/pipeline: add l2fwd with MAC swap example 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (37 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 38/40] examples/pipeline: add l2fwd example Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example Cristian Dumitrescu 39 siblings, 0 replies; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add L2 Forwarding example with MAC destination and source address swap to the pipeline application. Example command line: ./build/pipeline -l0-1 -- -s ./examples/l2fwd_macswp.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 2 + examples/pipeline/cli.c | 4 + examples/pipeline/example_l2fwd_macswp.c | 146 ++++++++++++++++++ examples/pipeline/examples/l2fwd_macswp.cli | 25 +++ .../pipeline/examples/l2fwd_macswp_pcap.cli | 20 +++ examples/pipeline/meson.build | 1 + 6 files changed, 198 insertions(+) create mode 100644 examples/pipeline/example_l2fwd_macswp.c create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index 405978ced..df5176296 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -11,6 +11,8 @@ SRCS-y += main.c SRCS-y += obj.c SRCS-y += thread.c SRCS-y += example_l2fwd.c +SRCS-y += example_l2fwd_macswp.c + # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index a54cd79ff..9547e1b4c 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -731,6 +731,8 @@ static const char cmd_pipeline_build_help[] = "pipeline <pipeline_name> build <app>\n"; int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); +int pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); + static void cmd_pipeline_build(char **tokens, uint32_t n_tokens, @@ -757,6 +759,8 @@ cmd_pipeline_build(char **tokens, app = tokens[3]; if (!strcmp(app, "l2fwd")) status = pipeline_setup_l2fwd(p->p); + else if (!strcmp(app, "l2fwd_macswp")) + status = pipeline_setup_l2fwd_macswp(p->p); else { snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); return; diff --git a/examples/pipeline/example_l2fwd_macswp.c b/examples/pipeline/example_l2fwd_macswp.c new file mode 100644 index 000000000..7922a75aa --- /dev/null +++ b/examples/pipeline/example_l2fwd_macswp.c @@ -0,0 +1,146 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_port_ethdev.h" +#include "rte_swx_port_source_sink.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ +static struct rte_swx_field_params ethernet_type[] = { + {"dst_addr", 48}, + {"src_addr", 48}, + {"ether_type", 16}, +}; + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_type[] = { + {"port", 32}, + {"addr", 48}, +}; + +/* + * Actions. + */ +static const char *action_macswp_instructions[] = { + "mov m.addr h.ethernet.dst_addr", + "mov h.ethernet.dst_addr h.ethernet.src_addr", + "mov h.ethernet.src_addr m.addr", + "return", +}; + +/* + * Tables. + */ +static const char *table_stub_actions[] = {"macswp"}; + +static struct rte_swx_pipeline_table_params table_stub_params = { + /* Match. */ + .fields = NULL, + .n_fields = 0, + + /* Action. */ + .action_names = table_stub_actions, + .n_actions = RTE_DIM(table_stub_actions), + .default_action_name = "macswp", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port", + "extract h.ethernet", + "table stub", + "xor m.port 1", + "emit h.ethernet", + "tx m.port", +}; + +int +pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); + +int +pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + err = rte_swx_pipeline_struct_type_register(p, + "ethernet_type", + ethernet_type, + RTE_DIM(ethernet_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ethernet", + "ethernet_type"); + CHECK(!err); + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_type", + metadata_type, + RTE_DIM(metadata_type)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_type"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "macswp", + NULL, + action_macswp_instructions, + RTE_DIM(action_macswp_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_config(p, + "stub", + &table_stub_params, + NULL, + NULL, + 0); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/l2fwd_macswp.cli b/examples/pipeline/examples/l2fwd_macswp.cli new file mode 100644 index 000000000..c5eb23d18 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp.cli @@ -0,0 +1,25 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 + +pipeline PIPELINE0 build l2fwd_macswp + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/l2fwd_macswp_pcap.cli b/examples/pipeline/examples/l2fwd_macswp_pcap.cli new file mode 100644 index 000000000..76c3071e3 --- /dev/null +++ b/examples/pipeline/examples/l2fwd_macswp_pcap.cli @@ -0,0 +1,20 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none + +pipeline PIPELINE0 build l2fwd_macswp + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index b16de3714..b13f04e01 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -16,4 +16,5 @@ sources = files( 'obj.c', 'thread.c', 'example_l2fwd.c', + 'example_l2fwd_macswp.c', ) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu ` (38 preceding siblings ...) 2020-08-26 15:14 ` [dpdk-dev] [PATCH 39/40] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu @ 2020-08-26 15:14 ` Cristian Dumitrescu 2020-08-26 17:05 ` Stephen Hemminger 39 siblings, 1 reply; 329+ messages in thread From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw) To: dev Add VXLAN encapsulation example to the pipeline application. The VXLAN tunnels can be generated with the vxlan.py script. Example command line: ./build/pipeline -l0-1 -- -s ./examples/vxlan.cli Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com> --- examples/pipeline/Makefile | 2 +- examples/pipeline/cli.c | 3 + examples/pipeline/example_vxlan.c | 318 ++++++++++++++++++++++ examples/pipeline/examples/vxlan.cli | 27 ++ examples/pipeline/examples/vxlan.py | 71 +++++ examples/pipeline/examples/vxlan.txt | 16 ++ examples/pipeline/examples/vxlan_pcap.cli | 22 ++ examples/pipeline/meson.build | 1 + 8 files changed, 459 insertions(+), 1 deletion(-) create mode 100644 examples/pipeline/example_vxlan.c create mode 100644 examples/pipeline/examples/vxlan.cli create mode 100644 examples/pipeline/examples/vxlan.py create mode 100644 examples/pipeline/examples/vxlan.txt create mode 100644 examples/pipeline/examples/vxlan_pcap.cli diff --git a/examples/pipeline/Makefile b/examples/pipeline/Makefile index df5176296..9e8088ec4 100644 --- a/examples/pipeline/Makefile +++ b/examples/pipeline/Makefile @@ -12,7 +12,7 @@ SRCS-y += obj.c SRCS-y += thread.c SRCS-y += example_l2fwd.c SRCS-y += example_l2fwd_macswp.c - +SRCS-y += example_vxlan.c # Build using pkg-config variables if possible ifeq ($(shell pkg-config --exists libdpdk && echo 0),0) diff --git a/examples/pipeline/cli.c b/examples/pipeline/cli.c index 9547e1b4c..d98400c78 100644 --- a/examples/pipeline/cli.c +++ b/examples/pipeline/cli.c @@ -732,6 +732,7 @@ static const char cmd_pipeline_build_help[] = int pipeline_setup_l2fwd(struct rte_swx_pipeline *p); int pipeline_setup_l2fwd_macswp(struct rte_swx_pipeline *p); +int pipeline_setup_vxlan(struct rte_swx_pipeline *p); static void cmd_pipeline_build(char **tokens, @@ -761,6 +762,8 @@ cmd_pipeline_build(char **tokens, status = pipeline_setup_l2fwd(p->p); else if (!strcmp(app, "l2fwd_macswp")) status = pipeline_setup_l2fwd_macswp(p->p); + else if (!strcmp(app, "vxlan")) + status = pipeline_setup_vxlan(p->p); else { snprintf(out, out_size, MSG_ARG_INVALID, tokens[0]); return; diff --git a/examples/pipeline/example_vxlan.c b/examples/pipeline/example_vxlan.c new file mode 100644 index 000000000..b4701e20f --- /dev/null +++ b/examples/pipeline/example_vxlan.c @@ -0,0 +1,318 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include <rte_common.h> + +#include "rte_swx_pipeline.h" +#include "rte_swx_table_em.h" + +#define CHECK(condition) \ +do { \ + if (!(condition)) { \ + printf("Error in function %s at line %d\n", \ + __FUNCTION__, __LINE__); \ + return -1; \ + } \ +} while (0) + +/* + * Packet headers. + */ +static struct rte_swx_field_params ethernet_h[] = { + {"dst_addr", 48}, + {"src_addr", 48}, + {"ether_type", 16}, +}; + +static struct rte_swx_field_params ipv4_h[] = { + {"ver_ihl", 8}, + {"diffserv", 8}, + {"total_len", 16}, + {"identification", 16}, + {"flags_offset", 16}, + {"ttl", 8}, + {"protocol", 8}, + {"hdr_checksum", 16}, + {"src_addr", 32}, + {"dst_addr", 32}, +}; + +static struct rte_swx_field_params udp_h[] = { + {"src_port", 16}, + {"dst_port", 16}, + {"length", 16}, + {"checksum", 16}, +}; + +static struct rte_swx_field_params vxlan_h[] = { + {"flags", 8}, + {"reserved", 24}, + {"vni", 24}, + {"reserved2", 8}, +}; + +/* + * Packet meta-data. + */ +static struct rte_swx_field_params metadata_t[] = { + {"port_in", 32}, + {"port_out", 32}, +}; + +/* + * Actions. + */ +static const char *drop_instructions[] = { + "mov m.port_out 4", + "tx m.port_out", +}; + +static struct rte_swx_field_params vxlan_encap_args_t[] = { + {"ethernet_dst_addr", 48}, + {"ethernet_src_addr", 48}, + {"ethernet_ether_type", 16}, + {"ipv4_ver_ihl", 8}, + {"ipv4_diffserv", 8}, + {"ipv4_total_len", 16}, + {"ipv4_identification", 16}, + {"ipv4_flags_offset", 16}, + {"ipv4_ttl", 8}, + {"ipv4_protocol", 8}, + {"ipv4_hdr_checksum", 16}, + {"ipv4_src_addr", 32}, + {"ipv4_dst_addr", 32}, + {"udp_src_port", 16}, + {"udp_dst_port", 16}, + {"udp_length", 16}, + {"udp_checksum", 16}, + {"vxlan_flags", 8}, + {"vxlan_reserved", 24}, + {"vxlan_vni", 24}, + {"vxlan_reserved2", 8}, + {"port_out", 32}, +}; + +/* Input frame: + * Ethernet (14) | IPv4 (total_len) + * + * Output frame: + * Ethernet (14) | IPv4 (20) | UDP (8) | VXLAN (8) | Input frame | FCS (4) + * + * Note: The input frame has its FCS removed before encapsulation in the output + * frame. + * + * Assumption: When read from the table, the outer IPv4 and UDP headers contain + * the following fields: + * - t.ipv4_total_len: Set to 50, which covers the length of: + * - The outer IPv4 header (20 bytes); + * - The outer UDP header (8 bytes); + * - The outer VXLAN header (8 bytes); + * - The inner Ethernet header (14 bytes); + * - t.ipv4_hdr_checksum: Includes the above total length. + * - t.udp_length: Set to 30, which covers the length of: + * - The outer UDP header (8 bytes); + * - The outer VXLAN header (8 bytes); + * - The inner Ethernet header (14 bytes); + * - t.udp_checksum: Set to 0. + * + * Once the total length of the inner IPv4 packet (h.ipv4.total_len) is known, + * the outer IPv4 and UDP headers are updated as follows: + * - h.outer_ipv4.total_len = t.ipv4_total_len + h.ipv4.total_len + * - h.outer_ipv4.hdr_checksum = t.ipv4_hdr_checksum + h.ipv4.total_len + * - h.outer_udp.length = t.udp_length + h.ipv4.total_len + * - h.outer_udp.checksum: No change. + */ +static const char *vxlan_encap_instructions[] = { + /* Copy from table entry to haders and metadata. */ + "dma h.outer_ethernet t.ethernet_dst_addr", + "dma h.outer_ipv4 t.ipv4_ver_ihl", + "dma h.outer_udp t.udp_src_port", + "dma h.outer_vxlan t.vxlan_flags", + "mov m.port_out t.port_out", + + /* Update h.outer_ipv4.total_len field. */ + "add h.outer_ipv4.total_len h.ipv4.total_len", + + /* Update h.outer_ipv4.hdr_checksum field. */ + "ckadd h.outer_ipv4.hdr_checksum h.ipv4.total_len", + + /* Update h.outer_udp.length field. */ + "add h.outer_udp.length h.ipv4.total_len", + + "return" +}; + +/* + * Tables. + */ +static struct rte_swx_match_field_params table_match_fields[] = { + [0] = { + .name = "h.ethernet.dst_addr", + .match_type = RTE_SWX_TABLE_MATCH_EXACT, + }, +}; + +static const char *table_actions[] = {"drop", "vxlan_encap"}; + +static struct rte_swx_pipeline_table_params table_params = { + /* Match. */ + .fields = table_match_fields, + .n_fields = RTE_DIM(table_match_fields), + + /* Action. */ + .action_names = table_actions, + .n_actions = RTE_DIM(table_actions), + .default_action_name = "drop", + .default_action_data = NULL, + .default_action_is_const = 0, +}; + +/* + * Pipeline. + */ +static const char *pipeline_instructions[] = { + "rx m.port_in", + "extract h.ethernet", + "extract h.ipv4", + "table vxlan", + "emit h.outer_ethernet", + "emit h.outer_ipv4", + "emit h.outer_udp", + "emit h.outer_vxlan", + "emit h.ethernet", + "emit h.ipv4", + "tx m.port_out", +}; + +int +pipeline_setup_vxlan(struct rte_swx_pipeline *p); + +int +pipeline_setup_vxlan(struct rte_swx_pipeline *p) +{ + int err; + + /* + * Packet headers. + */ + err = rte_swx_pipeline_struct_type_register(p, + "ethernet_h", + ethernet_h, + RTE_DIM(ethernet_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "ipv4_h", + ipv4_h, + RTE_DIM(ipv4_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "udp_h", + udp_h, + RTE_DIM(udp_h)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "vxlan_h", + vxlan_h, + RTE_DIM(vxlan_h)); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_ethernet", + "ethernet_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_ipv4", + "ipv4_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_udp", + "udp_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "outer_vxlan", + "vxlan_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ethernet", + "ethernet_h"); + CHECK(!err); + + err = rte_swx_pipeline_packet_header_register(p, + "ipv4", + "ipv4_h"); + CHECK(!err); + + /* + * Packet meta-data. + */ + err = rte_swx_pipeline_struct_type_register(p, + "metadata_t", + metadata_t, + RTE_DIM(metadata_t)); + CHECK(!err); + + err = rte_swx_pipeline_packet_metadata_register(p, + "metadata_t"); + CHECK(!err); + + /* + * Actions. + */ + err = rte_swx_pipeline_action_config(p, + "drop", + NULL, + drop_instructions, + RTE_DIM(drop_instructions)); + CHECK(!err); + + err = rte_swx_pipeline_struct_type_register(p, + "vxlan_encap_args_t", + vxlan_encap_args_t, + RTE_DIM(vxlan_encap_args_t)); + CHECK(!err); + + err = rte_swx_pipeline_action_config(p, + "vxlan_encap", + "vxlan_encap_args_t", + vxlan_encap_instructions, + RTE_DIM(vxlan_encap_instructions)); + CHECK(!err); + + /* + * Tables. + */ + err = rte_swx_pipeline_table_type_register(p, + "exact", + RTE_SWX_TABLE_MATCH_EXACT, + &rte_swx_table_exact_match_ops); + CHECK(!err); + + err = rte_swx_pipeline_table_config(p, + "vxlan", + &table_params, + NULL, + NULL, + 1 * 1024 * 1024); + CHECK(!err); + + /* + * Pipeline. + */ + err = rte_swx_pipeline_instructions_config(p, + pipeline_instructions, + RTE_DIM(pipeline_instructions)); + CHECK(!err); + + return 0; +} diff --git a/examples/pipeline/examples/vxlan.cli b/examples/pipeline/examples/vxlan.cli new file mode 100644 index 000000000..53a8b2a0a --- /dev/null +++ b/examples/pipeline/examples/vxlan.cli @@ -0,0 +1,27 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +link LINK0 dev 0000:18:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK1 dev 0000:18:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK2 dev 0000:3b:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK3 dev 0000:3b:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 link LINK0 rxq 0 bsz 32 +pipeline PIPELINE0 port in 1 link LINK1 rxq 0 bsz 32 +pipeline PIPELINE0 port in 2 link LINK2 rxq 0 bsz 32 +pipeline PIPELINE0 port in 3 link LINK3 rxq 0 bsz 32 + +pipeline PIPELINE0 port out 0 link LINK0 txq 0 bsz 32 +pipeline PIPELINE0 port out 1 link LINK1 txq 0 bsz 32 +pipeline PIPELINE0 port out 2 link LINK2 txq 0 bsz 32 +pipeline PIPELINE0 port out 3 link LINK3 txq 0 bsz 32 +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build vxlan +pipeline PIPELINE0 table vxlan update ./examples/vxlan.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/examples/vxlan.py b/examples/pipeline/examples/vxlan.py new file mode 100644 index 000000000..179d31b53 --- /dev/null +++ b/examples/pipeline/examples/vxlan.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python2 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +# + +from __future__ import print_function +import argparse +import re +import os + +DESCRIPTION = 'Table Generator' + +KEY = '0xaabbccdd{0:04x}' +ACTION = 'vxlan_encap' +ETHERNET_HEADER = 'ethernet_dst_addr N(0xa0a1a2a3{0:04x}) ' \ + 'ethernet_src_addr N(0xb0b1b2b3{0:04x}) ' \ + 'ethernet_ether_type N(0x0800)' +IPV4_HEADER = 'ipv4_ver_ihl N(0x45) ' \ + 'ipv4_diffserv N(0) ' \ + 'ipv4_total_len N(50) ' \ + 'ipv4_identification N(0) ' \ + 'ipv4_flags_offset N(0) ' \ + 'ipv4_ttl N(64) ' \ + 'ipv4_protocol N(17) ' \ + 'ipv4_hdr_checksum N(0x{1:04x}) ' \ + 'ipv4_src_addr N(0xc0c1{0:04x}) ' \ + 'ipv4_dst_addr N(0xd0d1{0:04x})' +UDP_HEADER = 'udp_src_port N(0xe0{0:02x}) ' \ + 'udp_dst_port N(4789) ' \ + 'udp_length N(30) ' \ + 'udp_checksum N(0)' +VXLAN_HEADER = 'vxlan_flags N(0) ' \ + 'vxlan_reserved N(0) ' \ + 'vxlan_vni N({0:d}) ' \ + 'vxlan_reserved2 N(0)' +PORT_OUT = 'port_out H({0:d})' + +def ipv4_header_checksum(i): + cksum = (0x4500 + 0x0032) + (0x0000 + 0x0000) + (0x4011 + 0x0000) + (0xc0c1 + i) + (0xd0d1 + i) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = (cksum & 0xFFFF) + (cksum >> 16) + cksum = ~cksum & 0xFFFF + return cksum + +def table_generate(n, p): + for i in range(0, n): + print("match %s action %s %s %s %s %s %s" % (KEY.format(i), + ACTION, + ETHERNET_HEADER.format(i), + IPV4_HEADER.format(i, ipv4_header_checksum(i)), + UDP_HEADER.format(i % 256), + VXLAN_HEADER.format(i), + PORT_OUT.format(i % p))) + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION) + + parser.add_argument( + '-n', + help='number of table entries (default: 65536)', + required=False, + default=65536) + + parser.add_argument( + '-p', + help='number of network ports (default: 4)', + required=False, + default=4) + + args = parser.parse_args() + table_generate(int(args.n), int(args.p)) diff --git a/examples/pipeline/examples/vxlan.txt b/examples/pipeline/examples/vxlan.txt new file mode 100644 index 000000000..acac80a38 --- /dev/null +++ b/examples/pipeline/examples/vxlan.txt @@ -0,0 +1,16 @@ +match 0xaabbccdd0000 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30000) ethernet_src_addr N(0xb0b1b2b30000) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe928) ipv4_src_addr N(0xc0c10000) ipv4_dst_addr N(0xd0d10000) udp_src_port N(0xe000) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(0) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0001 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30001) ethernet_src_addr N(0xb0b1b2b30001) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe926) ipv4_src_addr N(0xc0c10001) ipv4_dst_addr N(0xd0d10001) udp_src_port N(0xe001) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(1) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0002 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30002) ethernet_src_addr N(0xb0b1b2b30002) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe924) ipv4_src_addr N(0xc0c10002) ipv4_dst_addr N(0xd0d10002) udp_src_port N(0xe002) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(2) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0003 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30003) ethernet_src_addr N(0xb0b1b2b30003) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe922) ipv4_src_addr N(0xc0c10003) ipv4_dst_addr N(0xd0d10003) udp_src_port N(0xe003) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(3) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0004 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30004) ethernet_src_addr N(0xb0b1b2b30004) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe920) ipv4_src_addr N(0xc0c10004) ipv4_dst_addr N(0xd0d10004) udp_src_port N(0xe004) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(4) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0005 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30005) ethernet_src_addr N(0xb0b1b2b30005) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91e) ipv4_src_addr N(0xc0c10005) ipv4_dst_addr N(0xd0d10005) udp_src_port N(0xe005) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(5) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd0006 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30006) ethernet_src_addr N(0xb0b1b2b30006) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91c) ipv4_src_addr N(0xc0c10006) ipv4_dst_addr N(0xd0d10006) udp_src_port N(0xe006) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(6) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd0007 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30007) ethernet_src_addr N(0xb0b1b2b30007) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe91a) ipv4_src_addr N(0xc0c10007) ipv4_dst_addr N(0xd0d10007) udp_src_port N(0xe007) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(7) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd0008 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30008) ethernet_src_addr N(0xb0b1b2b30008) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe918) ipv4_src_addr N(0xc0c10008) ipv4_dst_addr N(0xd0d10008) udp_src_port N(0xe008) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(8) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd0009 action vxlan_encap ethernet_dst_addr N(0xa0a1a2a30009) ethernet_src_addr N(0xb0b1b2b30009) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe916) ipv4_src_addr N(0xc0c10009) ipv4_dst_addr N(0xd0d10009) udp_src_port N(0xe009) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(9) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000a action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000a) ethernet_src_addr N(0xb0b1b2b3000a) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe914) ipv4_src_addr N(0xc0c1000a) ipv4_dst_addr N(0xd0d1000a) udp_src_port N(0xe00a) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(10) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000b action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000b) ethernet_src_addr N(0xb0b1b2b3000b) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe912) ipv4_src_addr N(0xc0c1000b) ipv4_dst_addr N(0xd0d1000b) udp_src_port N(0xe00b) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(11) vxlan_reserved2 N(0) port_out H(3) +match 0xaabbccdd000c action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000c) ethernet_src_addr N(0xb0b1b2b3000c) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe910) ipv4_src_addr N(0xc0c1000c) ipv4_dst_addr N(0xd0d1000c) udp_src_port N(0xe00c) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(12) vxlan_reserved2 N(0) port_out H(0) +match 0xaabbccdd000d action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000d) ethernet_src_addr N(0xb0b1b2b3000d) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90e) ipv4_src_addr N(0xc0c1000d) ipv4_dst_addr N(0xd0d1000d) udp_src_port N(0xe00d) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(13) vxlan_reserved2 N(0) port_out H(1) +match 0xaabbccdd000e action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000e) ethernet_src_addr N(0xb0b1b2b3000e) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90c) ipv4_src_addr N(0xc0c1000e) ipv4_dst_addr N(0xd0d1000e) udp_src_port N(0xe00e) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(14) vxlan_reserved2 N(0) port_out H(2) +match 0xaabbccdd000f action vxlan_encap ethernet_dst_addr N(0xa0a1a2a3000f) ethernet_src_addr N(0xb0b1b2b3000f) ethernet_ether_type N(0x0800) ipv4_ver_ihl N(0x45) ipv4_diffserv N(0) ipv4_total_len N(50) ipv4_identification N(0) ipv4_flags_offset N(0) ipv4_ttl N(64) ipv4_protocol N(17) ipv4_hdr_checksum N(0xe90a) ipv4_src_addr N(0xc0c1000f) ipv4_dst_addr N(0xd0d1000f) udp_src_port N(0xe00f) udp_dst_port N(4789) udp_length N(30) udp_checksum N(0) vxlan_flags N(0) vxlan_reserved N(0) vxlan_vni N(15) vxlan_reserved2 N(0) port_out H(3) diff --git a/examples/pipeline/examples/vxlan_pcap.cli b/examples/pipeline/examples/vxlan_pcap.cli new file mode 100644 index 000000000..c406e27d3 --- /dev/null +++ b/examples/pipeline/examples/vxlan_pcap.cli @@ -0,0 +1,22 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2010-2020 Intel Corporation + +mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0 + +pipeline PIPELINE0 create 0 + +pipeline PIPELINE0 port in 0 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 1 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 2 source MEMPOOL0 ./examples/packet.pcap +pipeline PIPELINE0 port in 3 source MEMPOOL0 ./examples/packet.pcap + +pipeline PIPELINE0 port out 0 sink none +pipeline PIPELINE0 port out 1 sink none +pipeline PIPELINE0 port out 2 sink none +pipeline PIPELINE0 port out 3 sink none +pipeline PIPELINE0 port out 4 sink none + +pipeline PIPELINE0 build vxlan +pipeline PIPELINE0 table vxlan update ./examples/vxlan.txt none none + +thread 1 pipeline PIPELINE0 enable diff --git a/examples/pipeline/meson.build b/examples/pipeline/meson.build index b13f04e01..7f6f5218b 100644 --- a/examples/pipeline/meson.build +++ b/examples/pipeline/meson.build @@ -17,4 +17,5 @@ sources = files( 'thread.c', 'example_l2fwd.c', 'example_l2fwd_macswp.c', + 'example_vxlan.c', ) -- 2.17.1 ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example 2020-08-26 15:14 ` [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example Cristian Dumitrescu @ 2020-08-26 17:05 ` Stephen Hemminger 2020-09-07 21:49 ` Dumitrescu, Cristian 0 siblings, 1 reply; 329+ messages in thread From: Stephen Hemminger @ 2020-08-26 17:05 UTC (permalink / raw) To: Cristian Dumitrescu; +Cc: dev On Wed, 26 Aug 2020 16:14:45 +0100 Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > +/* > + * Packet headers. > + */ > +static struct rte_swx_field_params ethernet_h[] = { > + {"dst_addr", 48}, > + {"src_addr", 48}, > + {"ether_type", 16}, > +}; > + Could these tables be made const? Looks like read-only data. ^ permalink raw reply [flat|nested] 329+ messages in thread
* Re: [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example 2020-08-26 17:05 ` Stephen Hemminger @ 2020-09-07 21:49 ` Dumitrescu, Cristian 0 siblings, 0 replies; 329+ messages in thread From: Dumitrescu, Cristian @ 2020-09-07 21:49 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Wednesday, August 26, 2020 6:05 PM > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap > example > > On Wed, 26 Aug 2020 16:14:45 +0100 > Cristian Dumitrescu <cristian.dumitrescu@intel.com> wrote: > > > +/* > > + * Packet headers. > > + */ > > +static struct rte_swx_field_params ethernet_h[] = { > > + {"dst_addr", 48}, > > + {"src_addr", 48}, > > + {"ether_type", 16}, > > +}; > > + > > Could these tables be made const? Looks like read-only data. Thanks, Stephen, I just sent V2 where the examples have been completely reworked. The examples are now described in a pipeline specification file (closely aligned with P4) parsed at initialization, as opposed of C code. Regards, Cristian ^ permalink raw reply [flat|nested] 329+ messages in thread
end of thread, other threads:[~2020-10-04 18:29 UTC | newest] Thread overview: 329+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-08-26 15:14 [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-09 19:05 ` Stephen Hemminger 2020-09-09 19:52 ` Dumitrescu, Cristian 2020-09-10 8:42 ` Bruce Richardson 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 01/41] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-09-23 18:24 ` Stephen Hemminger 2020-09-23 18:37 ` Dumitrescu, Cristian 2020-09-23 20:23 ` Stephen Hemminger 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 01/42] pipeline: add new SWX pipeline type Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 12/42] pipeline: add SWX move instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 26/42] pipeline: add SWX instruction description Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-10-01 10:19 ` [dpdk-dev] [PATCH v7 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-10-01 12:03 ` David Marchand 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-10-04 7:50 ` Raslan Darawsheh 2020-10-04 9:01 ` David Marchand 2020-10-04 10:47 ` Raslan Darawsheh 2020-10-04 18:28 ` Dumitrescu, Cristian 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 33/42] port: add ethernet device SWX port Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 34/42] port: add source and sink SWX ports Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 35/42] table: add exact match SWX table Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 36/42] examples/pipeline: add new example application Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-10-01 10:20 ` [dpdk-dev] [PATCH v7 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu 2020-10-01 17:15 ` [dpdk-dev] [PATCH v7 00/42] Pipeline alignment with the P4 language David Marchand 2020-10-01 17:22 ` Dumitrescu, Cristian 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 02/42] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 03/42] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 04/42] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 05/42] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 06/42] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 07/42] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 08/42] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 09/42] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 10/42] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 11/42] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 12/42] pipeline: add SWX move instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 13/42] pipeline: add SWX DMA instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 14/42] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 15/42] pipeline: introduce SWX subtract instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 16/42] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 17/42] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 18/42] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 19/42] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 20/42] pipeline: introduce SWX XOR instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 21/42] pipeline: introduce SWX SHL instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 22/42] pipeline: introduce SWX SHR instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 23/42] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 24/42] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-09-30 6:33 ` [dpdk-dev] [PATCH v6 25/42] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 26/42] pipeline: add SWX instruction description Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 27/42] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 28/42] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 29/42] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 30/42] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 31/42] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 32/42] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-10-04 7:48 ` Raslan Darawsheh 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 33/42] port: add ethernet device SWX port Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 34/42] port: add source and sink SWX ports Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 35/42] table: add exact match SWX table Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 36/42] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 37/42] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 38/42] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 39/42] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 40/42] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 41/42] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-09-30 6:34 ` [dpdk-dev] [PATCH v6 42/42] doc: add new SWX pipeline type to release notes Cristian Dumitrescu 2020-09-30 19:34 ` [dpdk-dev] [PATCH v6 00/42] Pipeline alignment with the P4 language David Marchand 2020-10-01 10:45 ` Dumitrescu, Cristian 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 09/41] pipeline: add SWX Rx and extract instructions Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 10/41] pipeline: add SWX Tx and emit instructions Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 12/41] pipeline: add SWX move instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 13/41] pipeline: add SWX DMA instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 15/41] pipeline: introduce SWX subtract instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 20/41] pipeline: introduce SWX XOR instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 21/41] pipeline: introduce SWX SHL instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 22/41] pipeline: introduce SWX SHR instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 25/41] pipeline: introduce SWX jump and return instructions Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 26/41] pipeline: add SWX instruction description Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 33/41] port: add ethernet device SWX port Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 34/41] port: add source and sink SWX ports Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 35/41] table: add exact match SWX table Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 36/41] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-23 18:26 ` Stephen Hemminger 2020-09-23 18:30 ` Dumitrescu, Cristian 2020-09-29 13:51 ` David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-09-29 13:51 ` David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-23 18:06 ` [dpdk-dev] [PATCH v5 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-09-29 14:08 ` [dpdk-dev] [PATCH v5 00/41] Pipeline alignment with the P4 language David Marchand 2020-09-30 6:50 ` Dumitrescu, Cristian 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 26/41] pipeline: add SWX instruction description Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 33/41] port: add ethernet device SWX port Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 34/41] port: add source and sink SWX ports Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 35/41] table: add exact match SWX table Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 36/41] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-10 15:26 ` [dpdk-dev] [PATCH v4 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-09-22 20:05 ` [dpdk-dev] [PATCH v4 00/41] Pipeline alignment with the P4 language Wang, Han2 2020-09-22 20:08 ` Dumitrescu, Cristian 2020-09-23 11:45 ` David Marchand 2020-09-23 16:07 ` Dumitrescu, Cristian 2020-09-23 16:28 ` David Marchand 2020-09-23 16:40 ` Thomas Monjalon 2020-09-23 16:49 ` Dumitrescu, Cristian 2020-09-23 19:02 ` Dumitrescu, Cristian 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu 2020-09-08 20:17 ` [dpdk-dev] [PATCH v3 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 26/41] pipeline: add SWX instruction description Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 33/41] port: add ethernet device SWX port Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 34/41] port: add source and sink SWX ports Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 35/41] table: add exact match SWX table Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 36/41] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-08 20:18 ` [dpdk-dev] [PATCH v3 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 02/41] pipeline: add SWX pipeline input port Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 03/41] pipeline: add SWX pipeline output port Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 04/41] pipeline: add SWX headers and meta-data Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 05/41] pipeline: add SWX extern objects and funcs Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 06/41] pipeline: add SWX pipeline action Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 07/41] pipeline: add SWX pipeline tables Cristian Dumitrescu 2020-09-07 21:39 ` [dpdk-dev] [PATCH v2 08/41] pipeline: add SWX pipeline instructions Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 09/41] pipeline: add SWX rx and extract instructions Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 10/41] pipeline: add SWX tx and emit instructions Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 11/41] pipeline: add header validate and invalidate SWX instructions Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 12/41] pipeline: add SWX mov instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 13/41] pipeline: add SWX dma instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 14/41] pipeline: introduce SWX add instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 15/41] pipeline: introduce SWX sub instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 16/41] pipeline: introduce SWX ckadd instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 17/41] pipeline: introduce SWX cksub instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 18/41] pipeline: introduce SWX and instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 19/41] pipeline: introduce SWX or instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 20/41] pipeline: introduce SWX xor instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 21/41] pipeline: introduce SWX shl instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 22/41] pipeline: introduce SWX shr instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 23/41] pipeline: introduce SWX table instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 24/41] pipeline: introduce SWX extern instruction Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 25/41] pipeline: introduce SWX jmp and return instructions Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 26/41] pipeline: add SWX instruction description Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 27/41] pipeline: add SWX instruction verifier Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 28/41] pipeline: add SWX instruction optimizer Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 29/41] pipeline: add SWX pipeline query API Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 30/41] pipeline: add SWX pipeline flush Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 31/41] pipeline: add SWX table update high level API Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 32/41] pipeline: add SWX pipeline specification file Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 33/41] port: add ethernet device SWX port Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 34/41] port: add source and sink SWX ports Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 35/41] table: add exact match SWX table Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 36/41] examples/pipeline: add new example application Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 37/41] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 38/41] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 39/41] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 40/41] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-09-07 21:40 ` [dpdk-dev] [PATCH v2 41/41] examples/pipeline: add VXLAN encapsulation example Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 02/40] pipeline: add input port Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 03/40] pipeline: add output port Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 04/40] pipeline: add headers and meta-data Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 05/40] pipeline: add extern objects and functions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 06/40] pipeline: add action Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 07/40] pipeline: add tables Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 08/40] pipeline: add pipeline instructions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 09/40] pipeline: add rx and extract instructions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 10/40] pipeline: add tx and emit instructions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 11/40] pipeline: add header validate and invalidate instructions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 12/40] pipeline: add mov instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 13/40] pipeline: add dma instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 14/40] pipeline: introduce add instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 15/40] pipeline: introduce sub instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 16/40] pipeline: introduce ckadd instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 17/40] pipeline: introduce cksub instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 18/40] pipeline: introduce and instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 19/40] pipeline: introduce or instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 20/40] pipeline: introduce xor instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 21/40] pipeline: introduce shl instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 22/40] pipeline: introduce shr instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 23/40] pipeline: introduce table instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 24/40] pipeline: introduce extern instruction Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 25/40] pipeline: introduce jmp and return instructions Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 26/40] pipeline: add instruction verifier Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 27/40] pipeline: add instruction optimizer Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 28/40] pipeline: add pipeline query API Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 29/40] pipeline: add pipeline flush Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 30/40] pipeline: add instruction description Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 31/40] pipeline: add table update high level API Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 32/40] port: add ethernet device port Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 33/40] port: add source and sink ports Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 34/40] table: add exact match table Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 35/40] examples/pipeline: add new example application Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 36/40] examples/pipeline: add message passing mechanism Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 37/40] examples/pipeline: add configuration commands Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 38/40] examples/pipeline: add l2fwd example Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 39/40] examples/pipeline: add l2fwd with MAC swap example Cristian Dumitrescu 2020-08-26 15:14 ` [dpdk-dev] [PATCH 40/40] examples/pipeline: add VXLAN encap example Cristian Dumitrescu 2020-08-26 17:05 ` Stephen Hemminger 2020-09-07 21:49 ` Dumitrescu, Cristian
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).