From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC020A00C3; Tue, 16 Aug 2022 01:17:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C002340691; Tue, 16 Aug 2022 01:17:41 +0200 (CEST) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id 204984067C for ; Tue, 16 Aug 2022 01:17:41 +0200 (CEST) Received: by mail-lf1-f45.google.com with SMTP id u3so12561295lfk.8 for ; Mon, 15 Aug 2022 16:17:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:from:to:cc; bh=bXDbpx2vPSTtS6s8jvpF26EC2sqx8omADHzptW1zWp0=; b=VbCVYKvYhclY0VYw7jxeQdfki2/U+sgTXlTtQxE06MngcwFUyIqaoEpDvmrc4P9JCl cuylIOCKPMozHEdf01I764RhLKn7cvpQwhnuPH7RC/nee/nv4iN9nW+IjBSZqHaN3U+8 LSzLAIbccuF93gsdcgPzu+AOv5t2Lts5NwAEJqd2eWV/yhPczf0AnbAS1ItR8PbqzbPE 7PRzqBhMkTqqAmPF7YnY4uIMHfb7JWmk06RLywXZt544ZAeuOCLxwUE+4jp6xenSXe42 fPso70rrp/Bot911EnbRcodExTePtMhLylv5aGO2RZWg6J7Rx0FdKAGyv6Rpw0CXx2dY awFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:x-gm-message-state:from:to:cc; bh=bXDbpx2vPSTtS6s8jvpF26EC2sqx8omADHzptW1zWp0=; b=5ET63eKa/0DNu0FvCDE25zJBJeriN8ckYsH3zTjbQBMYoY7GfSmuEgAS7gRl07IXbH LGnoUYOyVLM6uNNctWoc+HYMNGkfF61TNZM8Q8BNeK2GNzsVFWrWU546ALVag+GHfFrd GiQAZBKvXDgVRWjr+FoqQ996WA/a+MsqVF0qg0sQ70hFwbOk8aYDLSIlfhoTlCnGShUK /nJyAKm+MH9rtc2F/7MGRQ71uTTXm3nV+Du2B7uWInm8EQYdh8sWSbj9ySyyDF9rl45p 53bO/zfptB1bMrMoViVyl327Ubze63nTv4qhrtLSX2nxrjl878xqdME8FhnRv6dfD7Lr TZjg== X-Gm-Message-State: ACgBeo0jYSuEvpEOTUxVtQi28OwJT1U8x66ivmR3O3JJrSRpb5Lg7sCk SyYGjdPEXxHSlH0jqtAFKH5ODayBjkg= X-Google-Smtp-Source: AA6agR6TIQsuj6AoUiOtdIk3OBY7oXJaS3lF3ig2NChbajgsJEhuH+0mVR3/JIvRl3sHLcyYQcLUcA== X-Received: by 2002:a05:6512:3092:b0:48a:f4f1:97c2 with SMTP id z18-20020a056512309200b0048af4f197c2mr6376832lfd.76.1660605460258; Mon, 15 Aug 2022 16:17:40 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id c24-20020ac24158000000b0048af6d4e5fbsm1209048lfi.275.2022.08.15.16.17.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Aug 2022 16:17:39 -0700 (PDT) Date: Tue, 16 Aug 2022 02:17:38 +0300 From: Dmitry Kozlyuk To: dev@dpdk.org Subject: [RFC] Dynamic log/trace control via telemetry Message-ID: <20220816021738.5498f802@sovereign> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When debugging a live app, useful info can be obtained from logs or traces that were not enabled when it was started and it is undesirable to restart. Furthermore, unless the app authors have considered tracing, rte_trace_save() is only called on exit, i.e. a shutdown is required again. What if the telemetry socket gave the missing control? For example: --> /eal/log/set_level,debug {"/eal/log/set_level": {"status": "success"}} --> /eal/log/set_level,pmd.net.*:debug {"/eal/log/set_level": {"status": "success"}} --> /eal/log/set_level,lib.[a-d],debug {"/eal/log/set_level": {"status": "success"}} --> /eal/log/set_level,foobar {"/eal/log/set_level": {"status": "fail", "message": "Invalid log level: foobar"}} Tracing is more complicated because it requires resource allocation. It could be: /eal/trace/set_mode,discard /eal/trace/set_mode,overwrite rte_trace_mode_set() /eal/trace/enable_pattern, /eal/trace/enable_regex, /eal/trace/disable_pattern, /eal/trace/disable_regex, rte_trace_enable_pattern() rte_trace_enable_regex() /eal/trace/rearm Clear the trace buffer. Apply the new settings accumulated by the previous commands. /eal/trace/save rte_trace_save() An open question is how to deal with multi-process. Only the primary process listens to the telemetry socket. Log and trace settings are not shared between processes. OTOH, when enabling some log source, it is desirable to do in all processes. In general, telemetry is not for changing the state of the app, but logs and traces are diagnostic information that seems to be in-scope. A similar suggestion was not opposed recently: http://inbox.dpdk.org/dev/24c49429394294cfbf0d9c506b205029bac77c8b.1657890378.git.anatoly.burakov@intel.com/