[PATCH v6 2/3] log: separate logging functions out of EAL

[Bruce Richardson <bruce.richardson@intel.com>]

Move the logging capability to a separate library, free from EAL. Rename
files as appropriate, and use meson.build to select the correct file to
be built for each operating system, rather than having a subdir per-os.
Add new documentation section in programmers guide to cover logging in
more detail.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
---
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 25 files changed, 202 insertions(+), 46 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index 1a4210b948..40f7dd2877 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 93c8a031be..8033f6cebd 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index d89cd3edb6..ac91060992 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
diff --git a/doc/guides/prog_guide/log_lib.rst b/doc/guides/prog_guide/log_lib.rst
new file mode 100644
index 0000000000..706ddcfef3
--- /dev/null
+++ b/doc/guides/prog_guide/log_lib.rst
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2023 Intel Corporation.
+
+.. _log_library:
+
+Log Library
+============
+
+The DPDK Log library provides the logging functionality for other DPDK libraries and drivers.
+By default, in a Linux application, logs are sent to syslog and also to the console.
+On FreeBSD and Windows applications, logs are sent only to the console.
+However, the log function can be overridden by the user to use a different logging mechanism.
+
+Log Levels
+-----------
+
+Log messages from apps and libraries are reported with a given level of severity.
+These levels, specified in ``rte_log.h`` are (from most to least important):
+
+#. Emergency
+#. Alert
+#. Critical
+#. Error
+#. Warning
+#. Notice
+#. Information
+#. Debug
+
+At runtime, only messages of a configured level or above (i.e. of higher importance)
+will be emitted by the application to the log output.
+That level can be configured either by the application calling the relevant APIs from the logging library,
+or by the user passing the ``--log-level`` parameter to the EAL via the application.
+
+Setting Global Log Level
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To adjust the global log level for an application,
+just pass a numeric level or a level name to the ``--log-level`` EAL parameter.
+For example::
+
+	/path/to/app --log-level=error
+
+	/path/to/app --log-level=debug
+
+	/path/to/app --log-level=5   # warning
+
+Within an application, the log level can be similarly set using the ``rte_log_set_global_level`` API.
+
+Setting Log Level for a Component
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In some cases, for example, for debugging purposes,
+it may be desirable to increase or decrease the log level for only a specific component, or set of components.
+To facilitate this, the ``--log-level`` argument also accepts an, optionally wildcarded, component name,
+along with the desired level for that component.
+For example::
+
+	/path/to/app --log-level=lib.eal:crit
+
+	/path/to/app --log-level=lib.*:warning
+
+Within an application, the same result can be got using the ``rte_log_set_level_pattern()`` or ``rte_log_set_level_regex()`` APIs.
+
+Using Logging APIs to Generate Log Messages
+--------------------------------------------
+
+To output log messages, ``rte_log()`` API function should be used.
+As well as the log message, ``rte_log()`` takes two additional parameters:
+
+* The log level
+* The log component type
+
+The log level is a numeric value as discussed above.
+The component type is a unique id that identifies the particular DPDK component to the logging system.
+To get this id, each component needs to register itself at startup,
+using the macro ``RTE_LOG_REGISTER_DEFAULT``.
+This macro takes two parameters, with the second being the default log level for the component.
+The first parameter, called "type", the name of the "logtype", or "component type" variable used in the component.
+This variable will be defined by the macro, and should be passed as the second parameter in calls to ``rte_log()``.
+In general, most DPDK components define their own logging macros to simplify the calls to the log APIs.
+They do this by:
+
+* Hiding the component type parameter inside the macro so it never needs to be passed explicitly.
+* Using the log-level definitions given in ``rte_log.h`` to allow short textual names to be used in
+  place of the numeric log levels.
+
+The following code is taken from ``rte_dmadev.c`` and shows the log registration,
+and subsequent definition of a shortcut logging macro.
+It can be used as a template for any new components using DPDK logging.
+
+.. code:: C
+
+	RTE_LOG_REGISTER_DEFAULT(rte_dma_logtype, INFO);
+	#define RTE_DMA_LOG(level, ...) \
+		rte_log(RTE_LOG_ ## level, rte_dma_logtype, RTE_FMT("dma: " \
+			RTE_FMT_HEAD(__VA_ARGS__,) "\n", RTE_FMT_TAIL(__VA_ARGS__,)))
+
+.. note::
+
+	Because the log registration macro provides the logtype variable definition,
+	it should be placed near the top of the C file using it.
+	If not, the logtype variable should be defined as an "extern int" near the top of the file.
+
+	Similarly, if logging is to be done by multiple files in a component,
+	only one file should register the logtype via the macro,
+	and the logtype should be defined as an "extern int" in a common header file.
+	Any component-specific logging macro should similarly be defined in that header.
+
+Throughout the dmadev library, all logging calls are therefore of the form:
+
+.. code:: C
+
+	RTE_DMA_LOG(ERR, "Name can't be NULL");
+
+	RTE_DMA_LOG(WARNING, "Device %d already started", dev_id);
diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c
index 062f1d8d9c..d74b581567 100644
--- a/lib/eal/common/eal_common_options.c
+++ b/lib/eal/common/eal_common_options.c
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
diff --git a/lib/eal/common/eal_private.h b/lib/eal/common/eal_private.h
index d7f83776c1..5eadba4902 100644
--- a/lib/eal/common/eal_private.h
+++ b/lib/eal/common/eal_private.h
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
diff --git a/lib/eal/common/meson.build b/lib/eal/common/meson.build
index 917758cc65..22a626ba6f 100644
--- a/lib/eal/common/meson.build
+++ b/lib/eal/common/meson.build
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
diff --git a/lib/eal/include/meson.build b/lib/eal/include/meson.build
index b0db9b3b3a..a0463efac7 100644
--- a/lib/eal/include/meson.build
+++ b/lib/eal/include/meson.build
@@ -28,7 +28,6 @@ headers += files(
         'rte_launch.h',
         'rte_lcore.h',
         'rte_lock_annotations.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
index c6efd92014..2f9203c87d 100644
--- a/lib/eal/linux/eal.c
+++ b/lib/eal/linux/eal.c
@@ -50,10 +50,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
diff --git a/lib/eal/linux/meson.build b/lib/eal/linux/meson.build
index 5af456db9e..e99ebed256 100644
--- a/lib/eal/linux/meson.build
+++ b/lib/eal/linux/meson.build
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
diff --git a/lib/eal/meson.build b/lib/eal/meson.build
index 9aa941a5ae..0fb974c78b 100644
--- a/lib/eal/meson.build
+++ b/lib/eal/meson.build
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
diff --git a/lib/eal/version.map b/lib/eal/version.map
index bdb98cf479..7940431e5a 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -140,21 +140,6 @@ DPDK_24 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_24 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_24 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
diff --git a/lib/eal/windows/eal.c b/lib/eal/windows/eal.c
index 096cb98d1c..22dc7f6ee1 100644
--- a/lib/eal/windows/eal.c
+++ b/lib/eal/windows/eal.c
@@ -27,8 +27,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
diff --git a/lib/eal/windows/meson.build b/lib/eal/windows/meson.build
index e4b2427610..7756d417be 100644
--- a/lib/eal/windows/meson.build
+++ b/lib/eal/windows/meson.build
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
diff --git a/lib/kvargs/meson.build b/lib/kvargs/meson.build
index b746516965..7eae744a8f 100644
--- a/lib/kvargs/meson.build
+++ b/lib/kvargs/meson.build
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
diff --git a/lib/eal/common/eal_common_log.c b/lib/log/log.c
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..c3ede383b2 100644
--- a/lib/eal/common/eal_common_log.c
+++ b/lib/log/log.c
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
diff --git a/lib/log/log_freebsd.c b/lib/log/log_freebsd.c
new file mode 100644
index 0000000000..698d3c5423
--- /dev/null
+++ b/lib/log/log_freebsd.c
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
diff --git a/lib/eal/common/eal_log.h b/lib/log/log_internal.h
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
--- a/lib/eal/common/eal_log.h
+++ b/lib/log/log_internal.h
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
diff --git a/lib/eal/linux/eal_log.c b/lib/log/log_linux.c
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
--- a/lib/eal/linux/eal_log.c
+++ b/lib/log/log_linux.c
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
diff --git a/lib/eal/windows/eal_log.c b/lib/log/log_windows.c
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
--- a/lib/eal/windows/eal_log.c
+++ b/lib/log/log_windows.c
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
diff --git a/lib/log/meson.build b/lib/log/meson.build
new file mode 100644
index 0000000000..6baff83ee5
--- /dev/null
+++ b/lib/log/meson.build
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
diff --git a/lib/eal/include/rte_log.h b/lib/log/rte_log.h
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
diff --git a/lib/log/version.map b/lib/log/version.map
new file mode 100644
index 0000000000..0648f8831a
--- /dev/null
+++ b/lib/log/version.map
@@ -0,0 +1,34 @@
+DPDK_24 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
diff --git a/lib/meson.build b/lib/meson.build
index fac2f52cad..c5e4642337 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
diff --git a/lib/telemetry/meson.build b/lib/telemetry/meson.build
index f84c9aa3be..489d000047 100644
--- a/lib/telemetry/meson.build
+++ b/lib/telemetry/meson.build
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
-- 
2.39.2