From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61F3BA00C5; Thu, 28 Jul 2022 17:11:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F096742BA6; Thu, 28 Jul 2022 17:11:52 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id D8D164014F for ; Thu, 28 Jul 2022 17:11:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659021111; x=1690557111; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=4/9aCdo3lRnWNij6lXocXywN2nFXgusGOl2a2zbTYe0=; b=CPPALscq76M2ynzMMdZIPyUw9UOCatIvmiPsOXFRwyR58aqfRsuCLA3f Q1Lzwzr7LgVMs+n00YAQDgfLy8nJnrmctfLRu0SxVzcTu5hh3P0i0G5At MI54hq8hsxkSa/4vUVRo3BvXDPyYvdPEQmUyCXGnWviPbqj7ORcsoe9fr Fj+jeb+XRhfaXeEzXdEF2p7uWQGDrGjKMAvbvAolP7E9E55sSLVPKYNtw jqBqfkO/DzzUCvxJDZJtJQusUAHGUqpQHZlRZuMoSFsiX8xUtTWvj0qC5 PzzImprdex6W66Q9stf0hth36SyV407IE8Gjgb7EkWx9FyGmMEfXOxR0O w==; X-IronPort-AV: E=McAfee;i="6400,9594,10422"; a="288547343" X-IronPort-AV: E=Sophos;i="5.93,198,1654585200"; d="scan'208";a="288547343" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jul 2022 08:11:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,198,1654585200"; d="scan'208";a="633727092" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com.) ([10.237.223.157]) by orsmga001.jf.intel.com with ESMTP; 28 Jul 2022 08:11:48 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Subject: [PATCH V6 00/17] pipeline: pipeline configuration and build improvements Date: Thu, 28 Jul 2022 15:11:30 +0000 Message-Id: <20220728151147.603265-1-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220718130713.339003-1-cristian.dumitrescu@intel.com> References: <20220718130713.339003-1-cristian.dumitrescu@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The sequence of steps to build a SWX pipeline object are: 1. Create the specification file (pipeline.p4 -> pipeline.spec): This text file represents the pipeline program that is typically generated by the P4 compiler or sometimes manually created. 2. Generate the C source code file (pipeline.spec -> pipeline.c): The C file contains a C function for every pipeline action and several C functions for the pipeline main program. It allows the C compiler to generate optimized code by having access to the entire pipeline program rather than using the small pre-build functions corresponding to the individual instructions 3. Build the shared object library (pipeline.c -> pipeline.o -> pipeline.so). 4. Load the shared object library (pipeline.so): At initialization, the pipeline object is “patched” with the optimized C functions from the shared object library. Previously, steps 2., 3. and 4. were implemented under the hood by the pipeline library at the initialization time in a completely hard-coded and non-customizable way. The user was not able to select the C compiler (GCC was assumed), the compiler version, the build flags, the file locations, etc. The code generation (step 2.) and library build (step 3.) were done on-the-fly at init and could potentially fail for many setup related reasons. Now, this process is no longer done under the hood by the pipeline library and the individual steps are explicitly supported by the API functions. The code generation (step 2.) is done off-line at any time before the pipeline execution. The library build (step 3.) is also done off-line and is now fully customizable by the user, who is able to gracefully access the generated C code and decide on the various build options. We also take the opportunity to streamline the pipeline I/O port configuration by introducing an I/O specification file. Essentially, the pipeline is now configured and build based on two files: a) The shared object library file (pipeline.so): It defines how the packets are processed by the program through tables and actions; the same P4 program can be executed by many pipelines. b) The I/O specification file (pipeline.io): High level text file defining how the packets are received and transmitted by the pipeline; this is not part of the P4 program, which is completely agnostic about the pipeline I/O ports. This is defined differently for each pipeline object when initialized. Change log: V6: -Added the cover letter. No code changes. V5: -Fixed more style issues reported by the CI/CD. V4: -Fixed style issues reported by the CI/CD. V3: -Added support for the pipeline I/O specification file. V2: -Fixed style issues reported by the CI/CD. Cristian Dumitrescu (17): pipeline: add pipeline name pipeline: move specification data structures to internal header pipeline: add pipeline specification data structure pipeline: rework the specification file-based pipeline build pipeline: generate the code for pipeline specification structure pipeline: add support for pipeline I/O specification pipeline: add API for pipeline code generation pipeline: add API for shared library-based pipeline build examples/pipeline: add CLI command for pipeline code generation examples/pipeline: add CLI command for shared library build examples/pipeline: remove the obsolete pipeline create CLI command examples/pipeline: remove the obsolete port configuration CLI commands examples/pipeline: remove the obsolete mirroring configuration CLI command examples/pipeline: use the pipeline name query API examples/pipeline: rework the link CLI command examples/pipelines: remove obsolete tap CLI command examples/pipeline: call the code generation and build CLI commands examples/pipeline/cli.c | 1133 +++------ examples/pipeline/examples/ethdev.io | 27 + examples/pipeline/examples/fib.cli | 44 +- examples/pipeline/examples/hash_func.cli | 41 +- examples/pipeline/examples/l2fwd.cli | 44 +- examples/pipeline/examples/l2fwd_macswp.cli | 44 +- .../pipeline/examples/l2fwd_macswp_pcap.cli | 35 +- examples/pipeline/examples/l2fwd_pcap.cli | 35 +- examples/pipeline/examples/learner.cli | 43 +- examples/pipeline/examples/meter.cli | 58 +- examples/pipeline/examples/mirroring.cli | 46 +- examples/pipeline/examples/pcap.io | 27 + examples/pipeline/examples/recirculation.cli | 41 +- examples/pipeline/examples/registers.cli | 53 +- examples/pipeline/examples/selector.cli | 55 +- examples/pipeline/examples/varbit.cli | 41 +- examples/pipeline/examples/vxlan.cli | 48 +- examples/pipeline/examples/vxlan_pcap.cli | 39 +- examples/pipeline/obj.c | 172 +- examples/pipeline/obj.h | 46 - examples/pipeline/thread.c | 65 +- examples/pipeline/thread.h | 9 +- lib/pipeline/rte_swx_ctl.c | 99 + lib/pipeline/rte_swx_ctl.h | 15 + lib/pipeline/rte_swx_pipeline.c | 464 ++-- lib/pipeline/rte_swx_pipeline.h | 71 +- lib/pipeline/rte_swx_pipeline_internal.h | 2 + lib/pipeline/rte_swx_pipeline_spec.c | 2050 +++++++++++++++-- lib/pipeline/rte_swx_pipeline_spec.h | 280 +++ lib/pipeline/version.map | 7 +- 30 files changed, 3430 insertions(+), 1704 deletions(-) create mode 100644 examples/pipeline/examples/ethdev.io create mode 100644 examples/pipeline/examples/pcap.io create mode 100644 lib/pipeline/rte_swx_pipeline_spec.h -- 2.34.1