DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 00/40] Pipeline alignment with the P4 language
@ 2020-08-26 15:14 Cristian Dumitrescu
  2020-08-26 15:14 ` [dpdk-dev] [PATCH 01/40] pipeline: add pipeline Cristian Dumitrescu
                   ` (39 more replies)
  0 siblings, 40 replies; 329+ messages in thread
From: Cristian Dumitrescu @ 2020-08-26 15:14 UTC (permalink / raw)
  To: dev

This patch set introduces a new pipeline type that combines the DPDK
performance with the flexibility of the P4-16 language[1]. The new API
can be used either by itself to code a complete software switch (SWX)
or data plane app, or in combination with the open-source P4 compiler
P4C [2], potentially acting as a P4C back-end, thus allowing the P4
programs to be translated to the DPDK API and run on multi-core CPUs.

Main new features:

* Nothing is hard-wired, everything is dynamically defined: The packet
  headers (i.e. protocols), the packet meta-data, the actions, the
  tables and the pipeline itself are dynamically defined instead of
  having to be selected from a pre-defined set.

* Instructions: The actions and the life of the packet through the
  pipeline are defined with instructions that manipulate the pipeline
  objects mentioned above. The pipeline is the main function of the
  packet program, with actions as subroutines triggered by the tables.

* Call external plugins: Extern objects and functions can be defined
  to call functionality that cannot be efficiently implemented with
  the existing pipeline-oriented instruction set, such as: special
  error detecting/correcting codes, crypto, meters, stats arrays,
  heuristics, etc.

* Better control plane interaction: Transaction-oriented table update
  mechanism that supports multi-table atomic updates. Multiple tables
  can be updated in a single step with only the before and after table
  sets visible to the packets. Alignment with P4Runtime [3].

* Performance: Multiple packets are in-flight within the pipeline at
  any moment. Each packet is owned by a different time-sharing thread
  in run-to-completion, with the thread pausing before memory access
  operations such as packet I/O and table lookup to allow the memory
  prefetch to complete. The instructions are verified and translated
  at initialization time with no run-time impact. The instructions are
  also optimized to detect and "fuse" frequently used patterns into
  vector-like instructions transparently to the user.

[1] P4-16 specification: https://p4.org/p4-spec/docs/P4-16-v1.2.1.pdf
[2] P4-16 compiler: https://github.com/p4lang/p4c
[3] P4Runtime specification:
    https://p4.org/p4runtime/spec/v1.2.0/P4Runtime-Spec.pdf

Cristian Dumitrescu (40):
  pipeline: add pipeline
  pipeline: add input port
  pipeline: add output port
  pipeline: add headers and meta-data
  pipeline: add extern objects and functions
  pipeline: add action
  pipeline: add tables
  pipeline: add pipeline instructions
  pipeline: add rx and extract instructions
  pipeline: add tx and emit instructions
  pipeline: add header validate and invalidate instructions
  pipeline: add mov instruction
  pipeline: add dma instruction
  pipeline: introduce add instruction
  pipeline: introduce sub instruction
  pipeline: introduce ckadd instruction
  pipeline: introduce cksub instruction
  pipeline: introduce and instruction
  pipeline: introduce or instruction
  pipeline: introduce xor instruction
  pipeline: introduce shl instruction
  pipeline: introduce shr instruction
  pipeline: introduce table instruction
  pipeline: introduce extern instruction
  pipeline: introduce jmp and return instructions
  pipeline: add instruction verifier
  pipeline: add instruction optimizer
  pipeline: add pipeline query API
  pipeline: add pipeline flush
  pipeline: add instruction description
  pipeline: add table update high level API
  port: add ethernet device port
  port: add source and sink ports
  table: add exact match table
  examples/pipeline: add new example application
  examples/pipeline: add message passing mechanism
  examples/pipeline: add configuration commands
  examples/pipeline: add l2fwd example
  examples/pipeline: add l2fwd with MAC swap example
  examples/pipeline: add VXLAN encap example

 examples/Makefile                             |    1 +
 examples/meson.build                          |    1 +
 examples/pipeline/Makefile                    |   85 +
 examples/pipeline/cli.c                       | 1394 ++++
 examples/pipeline/cli.h                       |   19 +
 examples/pipeline/conn.c                      |  331 +
 examples/pipeline/conn.h                      |   50 +
 examples/pipeline/example_l2fwd.c             |  125 +
 examples/pipeline/example_l2fwd_macswp.c      |  146 +
 examples/pipeline/example_vxlan.c             |  318 +
 examples/pipeline/examples/l2fwd.cli          |   25 +
 examples/pipeline/examples/l2fwd_macswp.cli   |   25 +
 .../pipeline/examples/l2fwd_macswp_pcap.cli   |   20 +
 examples/pipeline/examples/l2fwd_pcap.cli     |   20 +
 examples/pipeline/examples/packet.txt         |  102 +
 examples/pipeline/examples/vxlan.cli          |   27 +
 examples/pipeline/examples/vxlan.py           |   71 +
 examples/pipeline/examples/vxlan.txt          |   16 +
 examples/pipeline/examples/vxlan_pcap.cli     |   22 +
 examples/pipeline/main.c                      |  193 +
 examples/pipeline/meson.build                 |   21 +
 examples/pipeline/obj.c                       |  470 ++
 examples/pipeline/obj.h                       |  131 +
 examples/pipeline/thread.c                    |  549 ++
 examples/pipeline/thread.h                    |   28 +
 lib/librte_pipeline/Makefile                  |    5 +
 lib/librte_pipeline/meson.build               |   13 +-
 lib/librte_pipeline/rte_pipeline_version.map  |   43 +-
 lib/librte_pipeline/rte_swx_ctl.c             | 1552 ++++
 lib/librte_pipeline/rte_swx_ctl.h             |  568 ++
 lib/librte_pipeline/rte_swx_extern.h          |   98 +
 lib/librte_pipeline/rte_swx_pipeline.c        | 7197 +++++++++++++++++
 lib/librte_pipeline/rte_swx_pipeline.h        |  685 ++
 lib/librte_port/Makefile                      |    5 +
 lib/librte_port/meson.build                   |    9 +-
 lib/librte_port/rte_port_version.map          |    5 +-
 lib/librte_port/rte_swx_port.h                |  202 +
 lib/librte_port/rte_swx_port_ethdev.c         |  313 +
 lib/librte_port/rte_swx_port_ethdev.h         |   54 +
 lib/librte_port/rte_swx_port_source_sink.c    |  335 +
 lib/librte_port/rte_swx_port_source_sink.h    |   57 +
 lib/librte_table/Makefile                     |    3 +
 lib/librte_table/meson.build                  |    7 +-
 lib/librte_table/rte_swx_table.h              |  295 +
 lib/librte_table/rte_swx_table_em.c           |  851 ++
 lib/librte_table/rte_swx_table_em.h           |   30 +
 lib/librte_table/rte_table_version.map        |    7 +
 47 files changed, 16516 insertions(+), 8 deletions(-)
 create mode 100644 examples/pipeline/Makefile
 create mode 100644 examples/pipeline/cli.c
 create mode 100644 examples/pipeline/cli.h
 create mode 100644 examples/pipeline/conn.c
 create mode 100644 examples/pipeline/conn.h
 create mode 100644 examples/pipeline/example_l2fwd.c
 create mode 100644 examples/pipeline/example_l2fwd_macswp.c
 create mode 100644 examples/pipeline/example_vxlan.c
 create mode 100644 examples/pipeline/examples/l2fwd.cli
 create mode 100644 examples/pipeline/examples/l2fwd_macswp.cli
 create mode 100644 examples/pipeline/examples/l2fwd_macswp_pcap.cli
 create mode 100644 examples/pipeline/examples/l2fwd_pcap.cli
 create mode 100644 examples/pipeline/examples/packet.txt
 create mode 100644 examples/pipeline/examples/vxlan.cli
 create mode 100644 examples/pipeline/examples/vxlan.py
 create mode 100644 examples/pipeline/examples/vxlan.txt
 create mode 100644 examples/pipeline/examples/vxlan_pcap.cli
 create mode 100644 examples/pipeline/main.c
 create mode 100644 examples/pipeline/meson.build
 create mode 100644 examples/pipeline/obj.c
 create mode 100644 examples/pipeline/obj.h
 create mode 100644 examples/pipeline/thread.c
 create mode 100644 examples/pipeline/thread.h
 create mode 100644 lib/librte_pipeline/rte_swx_ctl.c
 create mode 100644 lib/librte_pipeline/rte_swx_ctl.h
 create mode 100644 lib/librte_pipeline/rte_swx_extern.h
 create mode 100644 lib/librte_pipeline/rte_swx_pipeline.c
 create mode 100644 lib/librte_pipeline/rte_swx_pipeline.h
 create mode 100644 lib/librte_port/rte_swx_port.h
 create mode 100644 lib/librte_port/rte_swx_port_ethdev.c
 create mode 100644 lib/librte_port/rte_swx_port_ethdev.h
 create mode 100644 lib/librte_port/rte_swx_port_source_sink.c
 create mode 100644 lib/librte_port/rte_swx_port_source_sink.h
 create mode 100644 lib/librte_table/rte_swx_table.h
 create mode 100644 lib/librte_table/rte_swx_table_em.c
 create mode 100644 lib/librte_table/rte_swx_table_em.h

-- 
2.17.1


^ permalink raw reply	[flat|nested] 329+ messages in thread