From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E4797436AA; Fri, 8 Dec 2023 15:59:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 797F5402CD; Fri, 8 Dec 2023 15:59:58 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1CB60402CC for ; Fri, 8 Dec 2023 15:59:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1702047596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3jyNT2QrZd/DM8y4ci2DMsLdgNTiVw9I/H9+3lCoal8=; b=SX7IN0kbMOk7/GCdPg7mfCk1tfIPhrr6qaM2FbrvYVqfikMyxtyUtUSO6TLKaHT3jIJr7C U+zpkLl2o68vhrR0l6U44u3Na1/YAxUZgt0G1kPHwGOOtpo44fH0R3IfvJl07W7QBYZZ5E eAn0Kl32UFomfDjORx/Vu5HNCKYpMuE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-226-xzsoxKWmMA6ALzTwx1F8NA-1; Fri, 08 Dec 2023 09:59:54 -0500 X-MC-Unique: xzsoxKWmMA6ALzTwx1F8NA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1E4F3185A787; Fri, 8 Dec 2023 14:59:54 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.224.122]) by smtp.corp.redhat.com (Postfix) with ESMTP id 80E0AC18572; Fri, 8 Dec 2023 14:59:52 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@amd.com, bruce.richardson@intel.com, stephen@networkplumber.org, mb@smartsharesystems.com Subject: [RFC v2 00/14] Detect superfluous newline in logs Date: Fri, 8 Dec 2023 15:59:34 +0100 Message-ID: <20231208145950.2184940-1-david.marchand@redhat.com> In-Reply-To: <20231117131824.1977792-1-david.marchand@redhat.com> References: <20231117131824.1977792-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Getting readable and consistent logs is important when running a DPDK application, especially when troubleshooting. A common issue with logs is when a DPDK change do not add (or on the contrary add too many \n) in the format string. This issue would only get noticed when actually hitting this log (which may be a situation hard to reach). This series proposes to introduce a new RTE_LOG_LINE helper that is responsible for logging a one line message and spews a build error (with gcc) if any \n is part of the format string. Because this is still a RFC and a lot of changes are added in this v2, no ack from the v1 has been kept. Since the v1 discussion on the cover letter, I changed my mind, and made the choice to break existing logging helpers exported in the public API. The reasoning is that those should not be used in the first place: logs should be produced only by the library that registers the logtype. Some multiline logging for debugging and the test assert macros are still present, but in this case an explicit call to RTE_LOG() is done. This can be checked with a simple: $ git grep 'RTE_LOG(' -- lib/ :^lib/log/ lib/acl/acl_bld.c: RTE_LOG(DEBUG, ACL, "Build phase for ACL \"%s\":\n" lib/acl/acl_gen.c: RTE_LOG(DEBUG, ACL, "Gen phase for ACL \"%s\":\n" lib/bpf/bpf_validate.c: RTE_LOG(DEBUG, BPF, "%s(%p) stats:\n" lib/bpf/bpf_validate.c: RTE_LOG(DEBUG, BPF, lib/eal/common/eal_common_debug.c: RTE_LOG(CRIT, EAL, "Error - exiting with code: %d\n" lib/eal/include/rte_test.h: RTE_LOG(ERR, EAL, "Test assert %s line %d failed: " \ lib/ip_frag/ip_frag_common.h:#define IP_FRAG_LOG(lvl, fmt, args...) RTE_LOG(lvl, IPFRAG, fmt, ##args) lib/sched/rte_sched.c: RTE_LOG(DEBUG, SCHED, "Low level config for pipe profile %u:\n" lib/sched/rte_sched.c: RTE_LOG(DEBUG, SCHED, "Low level config for subport profile %u:\n" Changes since RFC v1: - rebased after Stephen log changes, - added more fixes as I was making progress on the topic, - added a check so dpdk developers stop using RTE_LOG(), - added preparation patches, like "lib: replace logging helpers", - converted all libraries, keeping some special cases with explicit calls to RTE_LOG, -- David Marchand David Marchand (14): hash: remove some dead code regexdev: fix logtype register lib: use dedicated logtypes lib: add newline in logs lib: remove redundant newline from logs eal/linux: remove log paraphrasing the doc bpf: remove log level in internal helper lib: simplify multilines log messages rcu: introduce a logging helper vhost: improve log for memory dumping configuration log: add a per line log helper lib: convert to per line logging lib: replace logging helpers lib: use per line logging in helpers devtools/checkpatches.sh | 8 + drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 2 +- lib/acl/acl_bld.c | 28 +- lib/acl/acl_gen.c | 8 +- lib/acl/rte_acl.c | 8 +- lib/acl/tb_mem.c | 4 +- lib/bbdev/rte_bbdev.c | 11 +- lib/bpf/bpf.c | 2 +- lib/bpf/bpf_convert.c | 16 +- lib/bpf/bpf_exec.c | 12 +- lib/bpf/bpf_impl.h | 5 +- lib/bpf/bpf_jit_arm64.c | 8 +- lib/bpf/bpf_jit_x86.c | 4 +- lib/bpf/bpf_load.c | 2 +- lib/bpf/bpf_load_elf.c | 24 +- lib/bpf/bpf_pkt.c | 4 +- lib/bpf/bpf_stub.c | 8 +- lib/bpf/bpf_validate.c | 44 +- lib/cfgfile/rte_cfgfile.c | 18 +- lib/compressdev/rte_compressdev_internal.h | 5 +- lib/compressdev/rte_compressdev_pmd.c | 4 +- lib/cryptodev/rte_cryptodev.c | 4 +- lib/cryptodev/rte_cryptodev.h | 16 +- lib/dispatcher/rte_dispatcher.c | 12 +- lib/dmadev/rte_dmadev.c | 8 +- lib/eal/common/eal_common_bus.c | 22 +- lib/eal/common/eal_common_class.c | 4 +- lib/eal/common/eal_common_config.c | 2 +- lib/eal/common/eal_common_debug.c | 6 +- lib/eal/common/eal_common_dev.c | 80 +- lib/eal/common/eal_common_devargs.c | 18 +- lib/eal/common/eal_common_dynmem.c | 34 +- lib/eal/common/eal_common_fbarray.c | 12 +- lib/eal/common/eal_common_interrupts.c | 38 +- lib/eal/common/eal_common_lcore.c | 26 +- lib/eal/common/eal_common_memalloc.c | 12 +- lib/eal/common/eal_common_memory.c | 66 +- lib/eal/common/eal_common_memzone.c | 24 +- lib/eal/common/eal_common_options.c | 236 +++--- lib/eal/common/eal_common_proc.c | 112 +-- lib/eal/common/eal_common_tailqs.c | 12 +- lib/eal/common/eal_common_thread.c | 12 +- lib/eal/common/eal_common_timer.c | 6 +- lib/eal/common/eal_common_trace_utils.c | 2 +- lib/eal/common/eal_trace.h | 4 +- lib/eal/common/hotplug_mp.c | 54 +- lib/eal/common/malloc_elem.c | 6 +- lib/eal/common/malloc_heap.c | 40 +- lib/eal/common/malloc_mp.c | 72 +- lib/eal/common/rte_keepalive.c | 2 +- lib/eal/common/rte_malloc.c | 10 +- lib/eal/common/rte_service.c | 8 +- lib/eal/freebsd/eal.c | 74 +- lib/eal/freebsd/eal_alarm.c | 2 +- lib/eal/freebsd/eal_dev.c | 8 +- lib/eal/freebsd/eal_hugepage_info.c | 22 +- lib/eal/freebsd/eal_interrupts.c | 60 +- lib/eal/freebsd/eal_lcore.c | 2 +- lib/eal/freebsd/eal_memalloc.c | 10 +- lib/eal/freebsd/eal_memory.c | 34 +- lib/eal/freebsd/eal_thread.c | 2 +- lib/eal/freebsd/eal_timer.c | 10 +- lib/eal/linux/eal.c | 122 +-- lib/eal/linux/eal_alarm.c | 2 +- lib/eal/linux/eal_dev.c | 40 +- lib/eal/linux/eal_hugepage_info.c | 38 +- lib/eal/linux/eal_interrupts.c | 116 +-- lib/eal/linux/eal_lcore.c | 4 +- lib/eal/linux/eal_memalloc.c | 120 +-- lib/eal/linux/eal_memory.c | 208 ++--- lib/eal/linux/eal_thread.c | 4 +- lib/eal/linux/eal_timer.c | 14 +- lib/eal/linux/eal_vfio.c | 270 +++---- lib/eal/linux/eal_vfio_mp_sync.c | 4 +- lib/eal/riscv/rte_cycles.c | 4 +- lib/eal/unix/eal_filesystem.c | 14 +- lib/eal/unix/eal_firmware.c | 2 +- lib/eal/unix/eal_unix_memory.c | 8 +- lib/eal/unix/rte_thread.c | 34 +- lib/eal/windows/eal.c | 36 +- lib/eal/windows/eal_alarm.c | 12 +- lib/eal/windows/eal_debug.c | 8 +- lib/eal/windows/eal_dev.c | 8 +- lib/eal/windows/eal_hugepages.c | 10 +- lib/eal/windows/eal_interrupts.c | 10 +- lib/eal/windows/eal_lcore.c | 6 +- lib/eal/windows/eal_memalloc.c | 50 +- lib/eal/windows/eal_memory.c | 22 +- lib/eal/windows/eal_windows.h | 4 +- lib/eal/windows/include/rte_windows.h | 4 +- lib/eal/windows/rte_thread.c | 28 +- lib/efd/rte_efd.c | 58 +- lib/ethdev/ethdev_driver.c | 44 +- lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/ethdev_private.c | 10 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 858 ++++++++++----------- lib/ethdev/rte_ethdev.h | 51 +- lib/ethdev/rte_ethdev_cman.c | 16 +- lib/ethdev/rte_ethdev_telemetry.c | 44 +- lib/ethdev/rte_flow.c | 64 +- lib/ethdev/rte_flow.h | 3 - lib/ethdev/sff_telemetry.c | 30 +- lib/eventdev/eventdev_pmd.h | 14 +- lib/eventdev/rte_event_crypto_adapter.c | 12 +- lib/eventdev/rte_event_dma_adapter.c | 18 +- lib/eventdev/rte_event_eth_rx_adapter.c | 40 +- lib/eventdev/rte_event_eth_tx_adapter.c | 2 +- lib/eventdev/rte_event_timer_adapter.c | 21 +- lib/eventdev/rte_eventdev.c | 10 +- lib/fib/rte_fib.c | 14 +- lib/fib/rte_fib6.c | 14 +- lib/gpudev/gpudev.c | 6 +- lib/graph/graph_private.h | 5 +- lib/hash/rte_cuckoo_hash.c | 52 +- lib/hash/rte_cuckoo_hash.h | 11 - lib/hash/rte_fbk_hash.c | 4 +- lib/hash/rte_hash_crc.c | 12 +- lib/hash/rte_thash.c | 20 +- lib/hash/rte_thash_gfni.c | 8 +- lib/ip_frag/rte_ip_frag_common.c | 8 +- lib/latencystats/rte_latencystats.c | 41 +- lib/log/log.c | 6 +- lib/log/rte_log.h | 21 + lib/lpm/rte_lpm.c | 12 +- lib/lpm/rte_lpm6.c | 10 +- lib/mbuf/rte_mbuf.c | 14 +- lib/mbuf/rte_mbuf_dyn.c | 14 +- lib/mbuf/rte_mbuf_pool_ops.c | 4 +- lib/member/member.h | 14 + lib/member/rte_member.c | 15 +- lib/member/rte_member.h | 9 - lib/member/rte_member_heap.h | 39 +- lib/member/rte_member_ht.c | 13 +- lib/member/rte_member_sketch.c | 41 +- lib/member/rte_member_vbf.c | 9 +- lib/mempool/rte_mempool.c | 24 +- lib/mempool/rte_mempool.h | 2 +- lib/mempool/rte_mempool_ops.c | 10 +- lib/metrics/rte_metrics_telemetry.c | 6 +- lib/mldev/rte_mldev.c | 102 +-- lib/mldev/rte_mldev.h | 5 +- lib/net/rte_net_crc.c | 14 +- lib/node/ethdev_rx.c | 4 +- lib/node/ip4_lookup.c | 2 +- lib/node/ip6_lookup.c | 2 +- lib/node/kernel_rx.c | 8 +- lib/node/kernel_tx.c | 4 +- lib/node/node_private.h | 6 +- lib/pdump/rte_pdump.c | 113 ++- lib/pipeline/rte_pipeline.c | 228 +++--- lib/port/rte_port_ethdev.c | 18 +- lib/port/rte_port_eventdev.c | 18 +- lib/port/rte_port_fd.c | 24 +- lib/port/rte_port_frag.c | 14 +- lib/port/rte_port_ras.c | 12 +- lib/port/rte_port_ring.c | 18 +- lib/port/rte_port_sched.c | 12 +- lib/port/rte_port_source_sink.c | 48 +- lib/port/rte_port_sym_crypto.c | 18 +- lib/power/guest_channel.c | 36 +- lib/power/power_acpi_cpufreq.c | 116 +-- lib/power/power_amd_pstate_cpufreq.c | 132 ++-- lib/power/power_common.c | 14 +- lib/power/power_common.h | 6 +- lib/power/power_cppc_cpufreq.c | 130 ++-- lib/power/power_intel_uncore.c | 72 +- lib/power/power_kvm_vm.c | 22 +- lib/power/power_pstate_cpufreq.c | 156 ++-- lib/power/rte_power.c | 22 +- lib/power/rte_power_pmd_mgmt.c | 34 +- lib/power/rte_power_uncore.c | 14 +- lib/rawdev/rte_rawdev_pmd.h | 4 +- lib/rcu/rte_rcu_qsbr.c | 66 +- lib/rcu/rte_rcu_qsbr.h | 17 +- lib/regexdev/rte_regexdev.c | 88 +-- lib/regexdev/rte_regexdev.h | 13 +- lib/reorder/rte_reorder.c | 32 +- lib/rib/rte_rib.c | 10 +- lib/rib/rte_rib6.c | 10 +- lib/ring/rte_ring.c | 24 +- lib/sched/rte_pie.c | 18 +- lib/sched/rte_sched.c | 274 +++---- lib/stack/rte_stack.c | 8 +- lib/stack/stack_pvt.h | 4 +- lib/table/rte_table_acl.c | 72 +- lib/table/rte_table_array.c | 16 +- lib/table/rte_table_hash_cuckoo.c | 22 +- lib/table/rte_table_hash_ext.c | 22 +- lib/table/rte_table_hash_key16.c | 38 +- lib/table/rte_table_hash_key32.c | 38 +- lib/table/rte_table_hash_key8.c | 38 +- lib/table/rte_table_hash_lru.c | 22 +- lib/table/rte_table_lpm.c | 42 +- lib/table/rte_table_lpm_ipv6.c | 44 +- lib/table/rte_table_stub.c | 4 +- lib/telemetry/telemetry.c | 39 +- lib/vhost/fd_man.c | 8 +- lib/vhost/iotlb.c | 36 +- lib/vhost/socket.c | 102 +-- lib/vhost/vdpa.c | 8 +- lib/vhost/vduse.c | 120 +-- lib/vhost/vduse.h | 4 +- lib/vhost/vhost.c | 118 +-- lib/vhost/vhost.h | 24 +- lib/vhost/vhost_crypto.c | 12 +- lib/vhost/vhost_user.c | 530 ++++++------- lib/vhost/virtio_net.c | 188 ++--- lib/vhost/virtio_net_ctrl.c | 38 +- 209 files changed, 3982 insertions(+), 3963 deletions(-) create mode 100644 lib/member/member.h -- 2.43.0