From: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
To: dev@dpdk.org
Subject: [PATCH V6 00/17] pipeline: pipeline configuration and build improvements
Date: Thu, 28 Jul 2022 15:11:30 +0000 [thread overview]
Message-ID: <20220728151147.603265-1-cristian.dumitrescu@intel.com> (raw)
In-Reply-To: <20220718130713.339003-1-cristian.dumitrescu@intel.com>
The sequence of steps to build a SWX pipeline object are:
1. Create the specification file (pipeline.p4 -> pipeline.spec): This text file represents the
pipeline program that is typically generated by the P4 compiler or sometimes manually created.
2. Generate the C source code file (pipeline.spec -> pipeline.c): The C file contains a C function
for every pipeline action and several C functions for the pipeline main program. It allows the C
compiler to generate optimized code by having access to the entire pipeline program rather than
using the small pre-build functions corresponding to the individual instructions
3. Build the shared object library (pipeline.c -> pipeline.o -> pipeline.so).
4. Load the shared object library (pipeline.so): At initialization, the pipeline object is “patched”
with the optimized C functions from the shared object library.
Previously, steps 2., 3. and 4. were implemented under the hood by the pipeline library at the
initialization time in a completely hard-coded and non-customizable way. The user was not able to
select the C compiler (GCC was assumed), the compiler version, the build flags, the file locations,
etc. The code generation (step 2.) and library build (step 3.) were done on-the-fly at init and
could potentially fail for many setup related reasons.
Now, this process is no longer done under the hood by the pipeline library and the individual steps
are explicitly supported by the API functions. The code generation (step 2.) is done off-line at any
time before the pipeline execution. The library build (step 3.) is also done off-line and is now
fully customizable by the user, who is able to gracefully access the generated C code and decide on
the various build options.
We also take the opportunity to streamline the pipeline I/O port configuration by introducing an I/O
specification file. Essentially, the pipeline is now configured and build based on two files:
a) The shared object library file (pipeline.so): It defines how the packets are processed by the
program through tables and actions; the same P4 program can be executed by many pipelines.
b) The I/O specification file (pipeline.io): High level text file defining how the packets are
received and transmitted by the pipeline; this is not part of the P4 program, which is completely
agnostic about the pipeline I/O ports. This is defined differently for each pipeline object when
initialized.
Change log:
V6:
-Added the cover letter. No code changes.
V5:
-Fixed more style issues reported by the CI/CD.
V4:
-Fixed style issues reported by the CI/CD.
V3:
-Added support for the pipeline I/O specification file.
V2:
-Fixed style issues reported by the CI/CD.
Cristian Dumitrescu (17):
pipeline: add pipeline name
pipeline: move specification data structures to internal header
pipeline: add pipeline specification data structure
pipeline: rework the specification file-based pipeline build
pipeline: generate the code for pipeline specification structure
pipeline: add support for pipeline I/O specification
pipeline: add API for pipeline code generation
pipeline: add API for shared library-based pipeline build
examples/pipeline: add CLI command for pipeline code generation
examples/pipeline: add CLI command for shared library build
examples/pipeline: remove the obsolete pipeline create CLI command
examples/pipeline: remove the obsolete port configuration CLI commands
examples/pipeline: remove the obsolete mirroring configuration CLI
command
examples/pipeline: use the pipeline name query API
examples/pipeline: rework the link CLI command
examples/pipelines: remove obsolete tap CLI command
examples/pipeline: call the code generation and build CLI commands
examples/pipeline/cli.c | 1133 +++------
examples/pipeline/examples/ethdev.io | 27 +
examples/pipeline/examples/fib.cli | 44 +-
examples/pipeline/examples/hash_func.cli | 41 +-
examples/pipeline/examples/l2fwd.cli | 44 +-
examples/pipeline/examples/l2fwd_macswp.cli | 44 +-
.../pipeline/examples/l2fwd_macswp_pcap.cli | 35 +-
examples/pipeline/examples/l2fwd_pcap.cli | 35 +-
examples/pipeline/examples/learner.cli | 43 +-
examples/pipeline/examples/meter.cli | 58 +-
examples/pipeline/examples/mirroring.cli | 46 +-
examples/pipeline/examples/pcap.io | 27 +
examples/pipeline/examples/recirculation.cli | 41 +-
examples/pipeline/examples/registers.cli | 53 +-
examples/pipeline/examples/selector.cli | 55 +-
examples/pipeline/examples/varbit.cli | 41 +-
examples/pipeline/examples/vxlan.cli | 48 +-
examples/pipeline/examples/vxlan_pcap.cli | 39 +-
examples/pipeline/obj.c | 172 +-
examples/pipeline/obj.h | 46 -
examples/pipeline/thread.c | 65 +-
examples/pipeline/thread.h | 9 +-
lib/pipeline/rte_swx_ctl.c | 99 +
lib/pipeline/rte_swx_ctl.h | 15 +
lib/pipeline/rte_swx_pipeline.c | 464 ++--
lib/pipeline/rte_swx_pipeline.h | 71 +-
lib/pipeline/rte_swx_pipeline_internal.h | 2 +
lib/pipeline/rte_swx_pipeline_spec.c | 2050 +++++++++++++++--
lib/pipeline/rte_swx_pipeline_spec.h | 280 +++
lib/pipeline/version.map | 7 +-
30 files changed, 3430 insertions(+), 1704 deletions(-)
create mode 100644 examples/pipeline/examples/ethdev.io
create mode 100644 examples/pipeline/examples/pcap.io
create mode 100644 lib/pipeline/rte_swx_pipeline_spec.h
--
2.34.1
next prev parent reply other threads:[~2022-07-28 15:11 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-18 13:07 [PATCH 1/9] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 2/9] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 3/9] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 4/9] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 5/9] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 6/9] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 7/9] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 8/9] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-18 13:07 ` [PATCH 9/9] examples/pipeline: call CLI commands for code generation and build Cristian Dumitrescu
2022-07-18 13:25 ` [PATCH V2 1/9] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-18 13:25 ` [PATCH V2 2/9] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-18 13:25 ` [PATCH V2 3/9] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-18 13:25 ` [PATCH V2 4/9] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-18 13:25 ` [PATCH V2 5/9] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-18 13:26 ` [PATCH V2 6/9] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-18 13:26 ` [PATCH V2 7/9] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-18 13:26 ` [PATCH V2 8/9] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-18 13:26 ` [PATCH V2 9/9] examples/pipeline: call CLI commands for code generation and build Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 01/17] pipeline: add pipeline name Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 02/17] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 03/17] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 04/17] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 05/17] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 06/17] pipeline: add support for pipeline I/O specification Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 07/17] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 08/17] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 09/17] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 10/17] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 11/17] examples/pipeline: remove the obsolete pipeline create CLI command Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 12/17] examples/pipeline: remove the obsolete port configuration CLI commands Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 13/17] examples/pipeline: remove the obsolete mirroring configuration CLI command Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 14/17] examples/pipeline: use the pipeline name query API Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 15/17] examples/pipeline: rework the link CLI command Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 16/17] examples/pipelines: remove obsolete tap " Cristian Dumitrescu
2022-07-27 22:36 ` [PATCH V3 17/17] examples/pipeline: call the code generation and build CLI commands Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 01/17] pipeline: add pipeline name Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 02/17] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 03/17] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 04/17] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 05/17] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 06/17] pipeline: add support for pipeline I/O specification Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 07/17] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 08/17] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 09/17] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 10/17] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 11/17] examples/pipeline: remove the obsolete pipeline create CLI command Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 12/17] examples/pipeline: remove the obsolete port configuration CLI commands Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 13/17] examples/pipeline: remove the obsolete mirroring configuration CLI command Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 14/17] examples/pipeline: use the pipeline name query API Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 15/17] examples/pipeline: rework the link CLI command Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 16/17] examples/pipelines: remove obsolete tap " Cristian Dumitrescu
2022-07-27 22:54 ` [PATCH V4 17/17] examples/pipeline: call the code generation and build CLI commands Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 01/17] pipeline: add pipeline name Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 02/17] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 03/17] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 04/17] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 05/17] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 06/17] pipeline: add support for pipeline I/O specification Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 07/17] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 08/17] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 09/17] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 10/17] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 11/17] examples/pipeline: remove the obsolete pipeline create CLI command Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 12/17] examples/pipeline: remove the obsolete port configuration CLI commands Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 13/17] examples/pipeline: remove the obsolete mirroring configuration CLI command Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 14/17] examples/pipeline: use the pipeline name query API Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 15/17] examples/pipeline: rework the link CLI command Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 16/17] examples/pipelines: remove obsolete tap " Cristian Dumitrescu
2022-07-27 23:01 ` [PATCH V5 17/17] examples/pipeline: call the code generation and build CLI commands Cristian Dumitrescu
2022-07-28 8:22 ` [PATCH V5 01/17] pipeline: add pipeline name Bruce Richardson
2022-07-28 15:17 ` Dumitrescu, Cristian
2022-07-28 15:11 ` Cristian Dumitrescu [this message]
2022-07-28 15:11 ` [PATCH V6 " Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 02/17] pipeline: move specification data structures to internal header Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 03/17] pipeline: add pipeline specification data structure Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 04/17] pipeline: rework the specification file-based pipeline build Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 05/17] pipeline: generate the code for pipeline specification structure Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 06/17] pipeline: add support for pipeline I/O specification Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 07/17] pipeline: add API for pipeline code generation Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 08/17] pipeline: add API for shared library-based pipeline build Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 09/17] examples/pipeline: add CLI command for pipeline code generation Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 10/17] examples/pipeline: add CLI command for shared library build Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 11/17] examples/pipeline: remove the obsolete pipeline create CLI command Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 12/17] examples/pipeline: remove the obsolete port configuration CLI commands Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 13/17] examples/pipeline: remove the obsolete mirroring configuration CLI command Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 14/17] examples/pipeline: use the pipeline name query API Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 15/17] examples/pipeline: rework the link CLI command Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 16/17] examples/pipelines: remove obsolete tap " Cristian Dumitrescu
2022-07-28 15:11 ` [PATCH V6 17/17] examples/pipeline: call the code generation and build CLI commands Cristian Dumitrescu
2022-09-15 15:54 ` [PATCH V6 00/17] pipeline: pipeline configuration and build improvements Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220728151147.603265-1-cristian.dumitrescu@intel.com \
--to=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).