[dpdk-dev] [PATCH v10 01/18] eal: control max SIMD bitwidth

[David Marchand <david.marchand@redhat.com>]

From: Ciara Power <ciara.power@intel.com>

This patch adds a max SIMD bitwidth EAL configuration. The API allows
for an app to set this value. It can also be set using EAL argument
--force-max-simd-bitwidth, which will lock the value and override any
modifications made by the app.

Each arch has a define for the default SIMD bitwidth value, this is used
on EAL init to set the config max SIMD bitwidth.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
---
v10:
  - moved new API to rte_vect.h, prefixed with rte_vect_ and added
    experimental banner,
  - split patch 2 and squashed eal flags documentation in patch 1,
v9: Added release note.
v7: Reordered entries in the symbol .map file.
v4:
  - Used RTE_SIMD_MAX instead of UINT16_MAX.
  - Renamed enums to better reflect usage.
  - Added functions to windows symbol export file.
  - Modified Doxygen comments.
  - Modified enum name.
  - Changed RTE_SIMD_MAX value to a power of 2.
  - Merged patch 2 into this patch.
  - Enum now used for default value defines.
  - Fixed some small comments on v3.
v3:
  - Added enum value to essentially disable using max SIMD to choose
    paths, intended for use by ARM SVE.
  - Fixed parsing bitwidth argument to return an error for values
    greater than uint16_t.
  - Removed unnecessary define in generic rte_vect.h
  - Changed default bitwidth for ARM to UINT16_MAX, to allow for SVE.
v2:
  - Added to Doxygen comment for API.
  - Changed default bitwidth for Arm to 128.
---
 doc/guides/linux_gsg/eal_args.include.rst     | 16 +++++
 .../prog_guide/env_abstraction_layer.rst      | 34 ++++++++++
 doc/guides/rel_notes/release_20_11.rst        |  6 ++
 lib/librte_eal/arm/include/rte_vect.h         |  2 +
 lib/librte_eal/common/eal_common_options.c    | 66 +++++++++++++++++++
 lib/librte_eal/common/eal_internal_cfg.h      |  8 +++
 lib/librte_eal/common/eal_options.h           |  2 +
 lib/librte_eal/include/generic/rte_vect.h     | 53 ++++++++++++++-
 lib/librte_eal/ppc/include/rte_vect.h         |  2 +
 lib/librte_eal/rte_eal_exports.def            |  3 +
 lib/librte_eal/rte_eal_version.map            |  2 +
 lib/librte_eal/x86/include/rte_vect.h         |  2 +
 12 files changed, 194 insertions(+), 2 deletions(-)

diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst
index 0fe4457968..a0bfbd1a98 100644
--- a/doc/guides/linux_gsg/eal_args.include.rst
+++ b/doc/guides/linux_gsg/eal_args.include.rst
@@ -210,3 +210,19 @@ Other options
 *    ``--no-telemetry``:
 
     Disable telemetry.
+
+*    ``--force-max-simd-bitwidth=<val>``:
+
+    Specify the maximum SIMD bitwidth size to handle. This limits which vector paths,
+    if any, are taken, as any paths taken must use a bitwidth below the max bitwidth limit.
+    For example, to allow all SIMD bitwidths up to and including AVX-512::
+
+        --force-max-simd-bitwidth=512
+
+    The following example shows limiting the bitwidth to 64-bits to disable all vector code::
+
+        --force-max-simd-bitwidth=64
+
+    To disable use of max SIMD bitwidth limit::
+
+        --force-max-simd-bitwidth=0
diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 936c885081..fafed83fba 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -486,6 +486,40 @@ the desired addressing mode when virtual devices that are not directly attached
 To facilitate forcing the IOVA mode to a specific value the EAL command line option ``--iova-mode`` can
 be used to select either physical addressing('pa') or virtual addressing('va').
 
+.. _max_simd_bitwidth:
+
+
+Max SIMD bitwidth
+~~~~~~~~~~~~~~~~~
+
+The EAL provides a single setting to limit the max SIMD bitwidth used by DPDK,
+which is used in determining the vector path, if any, chosen by a component.
+The value can be set at runtime by an application using the
+'rte_vect_set_max_simd_bitwidth(uint16_t bitwidth)' function,
+which should only be called once at initialization, before EAL init.
+The value can be overridden by the user using the EAL command-line option '--force-max-simd-bitwidth'.
+
+When choosing a vector path, along with checking the CPU feature support,
+the value of the max SIMD bitwidth must also be checked, and can be retrieved using the
+'rte_vect_get_max_simd_bitwidth()' function.
+The value should be compared against the enum values for accepted max SIMD bitwidths:
+
+.. code-block:: c
+
+   enum rte_vect_max_simd {
+       RTE_VECT_SIMD_DISABLED = 64,
+       RTE_VECT_SIMD_128 = 128,
+       RTE_VECT_SIMD_256 = 256,
+       RTE_VECT_SIMD_512 = 512,
+       RTE_VECT_SIMD_MAX = INT16_MAX + 1,
+   };
+
+    if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_512)
+        /* Take AVX-512 vector path */
+    else if (rte_vect_get_max_simd_bitwidth() >= RTE_VECT_SIMD_256)
+        /* Take AVX2 vector path */
+
+
 Memory Segments and Memory Zones (memzone)
 ------------------------------------------
 
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 25ea3c20b4..c2bd6ee741 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -77,6 +77,12 @@ New Features
   This API is specific to x86 and implemented as a stub for other
   architectures.
 
+* **Added support for limiting maximum SIMD bitwidth.**
+
+  Added a new EAL config setting ``max_simd_bitwidth`` to limit the vector
+  path selection at runtime. This value can be set by apps using the
+  ``rte_vect_set_max_simd_bitwidth`` function, or by the user with EAL flag ``--force-max-simd-bitwidth``.
+
 * **Updated CRC modules of the net library.**
 
   * Added runtime selection of the optimal architecture-specific CRC path.
diff --git a/lib/librte_eal/arm/include/rte_vect.h b/lib/librte_eal/arm/include/rte_vect.h
index f6a455b4e7..a739e6e66a 100644
--- a/lib/librte_eal/arm/include/rte_vect.h
+++ b/lib/librte_eal/arm/include/rte_vect.h
@@ -14,6 +14,8 @@
 extern "C" {
 #endif
 
+#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_MAX
+
 typedef int32x4_t xmm_t;
 
 #define	XMM_SIZE	(sizeof(xmm_t))
diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index a5426e1234..60f0e4c69b 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -35,6 +35,7 @@
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include <rte_telemetry.h>
 #endif
+#include <rte_vect.h>
 
 #include "eal_internal_cfg.h"
 #include "eal_options.h"
@@ -102,6 +103,7 @@ eal_long_options[] = {
 	{OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM},
 	{OPT_TELEMETRY,         0, NULL, OPT_TELEMETRY_NUM        },
 	{OPT_NO_TELEMETRY,      0, NULL, OPT_NO_TELEMETRY_NUM     },
+	{OPT_FORCE_MAX_SIMD_BITWIDTH, 1, NULL, OPT_FORCE_MAX_SIMD_BITWIDTH_NUM},
 	{0,                     0, NULL, 0                        }
 };
 
@@ -343,6 +345,8 @@ eal_reset_internal_config(struct internal_config *internal_cfg)
 	internal_cfg->user_mbuf_pool_ops_name = NULL;
 	CPU_ZERO(&internal_cfg->ctrl_cpuset);
 	internal_cfg->init_complete = 0;
+	internal_cfg->max_simd_bitwidth.bitwidth = RTE_VECT_DEFAULT_SIMD_BITWIDTH;
+	internal_cfg->max_simd_bitwidth.forced = 0;
 }
 
 static int
@@ -1309,6 +1313,34 @@ eal_parse_iova_mode(const char *name)
 	return 0;
 }
 
+static int
+eal_parse_simd_bitwidth(const char *arg)
+{
+	char *end;
+	unsigned long bitwidth;
+	int ret;
+	struct internal_config *internal_conf =
+		eal_get_internal_configuration();
+
+	if (arg == NULL || arg[0] == '\0')
+		return -1;
+
+	errno = 0;
+	bitwidth = strtoul(arg, &end, 0);
+
+	/* check for errors */
+	if (errno != 0 || end == NULL || *end != '\0' || bitwidth > RTE_VECT_SIMD_MAX)
+		return -1;
+
+	if (bitwidth == 0)
+		bitwidth = (unsigned long) RTE_VECT_SIMD_MAX;
+	ret = rte_vect_set_max_simd_bitwidth(bitwidth);
+	if (ret < 0)
+		return -1;
+	internal_conf->max_simd_bitwidth.forced = 1;
+	return 0;
+}
+
 static int
 eal_parse_base_virtaddr(const char *arg)
 {
@@ -1707,6 +1739,13 @@ eal_parse_common_option(int opt, const char *optarg,
 	case OPT_NO_TELEMETRY_NUM:
 		conf->no_telemetry = 1;
 		break;
+	case OPT_FORCE_MAX_SIMD_BITWIDTH_NUM:
+		if (eal_parse_simd_bitwidth(optarg) < 0) {
+			RTE_LOG(ERR, EAL, "invalid parameter for --"
+					OPT_FORCE_MAX_SIMD_BITWIDTH "\n");
+			return -1;
+		}
+		break;
 
 	/* don't know what to do, leave this to caller */
 	default:
@@ -1903,6 +1942,32 @@ eal_check_common_options(struct internal_config *internal_cfg)
 	return 0;
 }
 
+uint16_t
+rte_vect_get_max_simd_bitwidth(void)
+{
+	const struct internal_config *internal_conf =
+		eal_get_internal_configuration();
+	return internal_conf->max_simd_bitwidth.bitwidth;
+}
+
+int
+rte_vect_set_max_simd_bitwidth(uint16_t bitwidth)
+{
+	struct internal_config *internal_conf =
+		eal_get_internal_configuration();
+	if (internal_conf->max_simd_bitwidth.forced) {
+		RTE_LOG(NOTICE, EAL, "Cannot set max SIMD bitwidth - user runtime override enabled");
+		return -EPERM;
+	}
+
+	if (bitwidth < RTE_VECT_SIMD_DISABLED || !rte_is_power_of_2(bitwidth)) {
+		RTE_LOG(ERR, EAL, "Invalid bitwidth value!\n");
+		return -EINVAL;
+	}
+	internal_conf->max_simd_bitwidth.bitwidth = bitwidth;
+	return 0;
+}
+
 void
 eal_common_usage(void)
 {
@@ -1981,6 +2046,7 @@ eal_common_usage(void)
 	       "  --"OPT_BASE_VIRTADDR"     Base virtual address\n"
 	       "  --"OPT_TELEMETRY"   Enable telemetry support (on by default)\n"
 	       "  --"OPT_NO_TELEMETRY"   Disable telemetry support\n"
+	       "  --"OPT_FORCE_MAX_SIMD_BITWIDTH" Force the max SIMD bitwidth\n"
 	       "\nEAL options for DEBUG use only:\n"
 	       "  --"OPT_HUGE_UNLINK"       Unlink hugepage files after init\n"
 	       "  --"OPT_NO_HUGE"           Use malloc instead of hugetlbfs\n"
diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h
index de627c7627..51dbe86e2b 100644
--- a/lib/librte_eal/common/eal_internal_cfg.h
+++ b/lib/librte_eal/common/eal_internal_cfg.h
@@ -33,6 +33,12 @@ struct hugepage_info {
 	int lock_descriptor;    /**< file descriptor for hugepage dir */
 };
 
+struct simd_bitwidth {
+	bool forced;
+	/**< flag indicating if bitwidth is forced and can't be modified */
+	uint16_t bitwidth; /**< bitwidth value */
+};
+
 /**
  * internal configuration
  */
@@ -85,6 +91,8 @@ struct internal_config {
 	volatile unsigned int init_complete;
 	/**< indicates whether EAL has completed initialization */
 	unsigned int no_telemetry; /**< true to disable Telemetry */
+	struct simd_bitwidth max_simd_bitwidth;
+	/**< max simd bitwidth path to use */
 };
 
 void eal_reset_internal_config(struct internal_config *internal_cfg);
diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h
index 89769d48b4..ef33979664 100644
--- a/lib/librte_eal/common/eal_options.h
+++ b/lib/librte_eal/common/eal_options.h
@@ -85,6 +85,8 @@ enum {
 	OPT_TELEMETRY_NUM,
 #define OPT_NO_TELEMETRY      "no-telemetry"
 	OPT_NO_TELEMETRY_NUM,
+#define OPT_FORCE_MAX_SIMD_BITWIDTH  "force-max-simd-bitwidth"
+	OPT_FORCE_MAX_SIMD_BITWIDTH_NUM,
 	OPT_LONG_MAX_NUM
 };
 
diff --git a/lib/librte_eal/include/generic/rte_vect.h b/lib/librte_eal/include/generic/rte_vect.h
index 3fc47979f8..736066d430 100644
--- a/lib/librte_eal/include/generic/rte_vect.h
+++ b/lib/librte_eal/include/generic/rte_vect.h
@@ -7,13 +7,16 @@
 
 /**
  * @file
- * SIMD vector types
+ * SIMD vector types and control
  *
- * This file defines types to use vector instructions with generic C code.
+ * This file defines types to use vector instructions with generic C code
+ * and APIs to enable the code using them.
  */
 
 #include <stdint.h>
 
+#include <rte_compat.h>
+
 /* Unsigned vector types */
 
 /**
@@ -183,4 +186,50 @@ typedef int32_t rte_v256s32_t __attribute__((vector_size(32), aligned(32)));
  */
 typedef int64_t rte_v256s64_t __attribute__((vector_size(32), aligned(32)));
 
+/**
+ * The max SIMD bitwidth value to limit vector path selection.
+ */
+enum rte_vect_max_simd {
+	RTE_VECT_SIMD_DISABLED = 64,
+	/**< Limits path selection to scalar, disables all vector paths. */
+	RTE_VECT_SIMD_128 = 128,
+	/**< Limits path selection to SSE/NEON/Altivec or below. */
+	RTE_VECT_SIMD_256 = 256, /**< Limits path selection to AVX2 or below. */
+	RTE_VECT_SIMD_512 = 512, /**< Limits path selection to AVX512 or below. */
+	RTE_VECT_SIMD_MAX = INT16_MAX + 1,
+	/**<
+	 * Disables limiting by max SIMD bitwidth, allows all suitable paths.
+	 * This value is used as it is a large number and a power of 2.
+	 */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Get the supported SIMD bitwidth.
+ *
+ * @return
+ *   uint16_t bitwidth.
+ */
+__rte_experimental
+uint16_t rte_vect_get_max_simd_bitwidth(void);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Set the supported SIMD bitwidth.
+ * This API should only be called once at initialization, before EAL init.
+ *
+ * @param bitwidth
+ *   uint16_t bitwidth.
+ * @return
+ *   - 0 on success.
+ *   - -EINVAL on invalid bitwidth parameter.
+ *   - -EPERM if bitwidth is forced.
+ */
+__rte_experimental
+int rte_vect_set_max_simd_bitwidth(uint16_t bitwidth);
+
 #endif /* _RTE_VECT_H_ */
diff --git a/lib/librte_eal/ppc/include/rte_vect.h b/lib/librte_eal/ppc/include/rte_vect.h
index b0545c878c..c1f0b0672c 100644
--- a/lib/librte_eal/ppc/include/rte_vect.h
+++ b/lib/librte_eal/ppc/include/rte_vect.h
@@ -15,6 +15,8 @@
 extern "C" {
 #endif
 
+#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_256
+
 typedef vector signed int xmm_t;
 
 #define	XMM_SIZE	(sizeof(xmm_t))
diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_exports.def
index 975acb8ffe..7eab9fbd2c 100644
--- a/lib/librte_eal/rte_eal_exports.def
+++ b/lib/librte_eal/rte_eal_exports.def
@@ -303,6 +303,9 @@ EXPORTS
 	rte_thread_register
 	rte_thread_unregister
 
+	rte_vect_get_max_simd_bitwidth
+	rte_vect_set_max_simd_bitwidth
+
 	rte_mem_lock
 	rte_mem_map
 	rte_mem_page_size
diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
index df8e0af758..a625ca2dc8 100644
--- a/lib/librte_eal/rte_eal_version.map
+++ b/lib/librte_eal/rte_eal_version.map
@@ -400,6 +400,8 @@ EXPERIMENTAL {
 	__rte_eal_trace_generic_size_t;
 	rte_epoll_wait_interruptible;
 	rte_service_lcore_may_be_active;
+	rte_vect_get_max_simd_bitwidth;
+	rte_vect_set_max_simd_bitwidth;
 };
 
 INTERNAL {
diff --git a/lib/librte_eal/x86/include/rte_vect.h b/lib/librte_eal/x86/include/rte_vect.h
index 64383c3606..2e40b77da9 100644
--- a/lib/librte_eal/x86/include/rte_vect.h
+++ b/lib/librte_eal/x86/include/rte_vect.h
@@ -36,6 +36,8 @@
 extern "C" {
 #endif
 
+#define RTE_VECT_DEFAULT_SIMD_BITWIDTH RTE_VECT_SIMD_256
+
 typedef __m128i xmm_t;
 
 #define	XMM_SIZE	(sizeof(xmm_t))
-- 
2.23.0