From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 41133A0560; Tue, 18 Oct 2022 15:14:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D9C4740395; Tue, 18 Oct 2022 15:14:51 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 9DC204021D for ; Tue, 18 Oct 2022 15:14:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666098889; x=1697634889; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=oH+IP2cAgu3wKhdKiiiFZCZAjV+TAMlrG0MKhA852tA=; b=kFrNEsHm36FLlH8/kEi+qu2eWEMnEYfM5gtu9XTW3nghXSJJfO1olY2V SEED/kKhD5pT2G9ZaShOyIFhMCm6imiYxWH5w3iqigRH2hkev3S7FHoPh E7YHTHY3QqQRCLmKfg8M9n73YTqPSiJzx74mJAOnrE1ltfHdfDF+vCFTa Fsswq4hYUzUM0JN9lkD94hJ6OoV1MaHybLNnfU7Pzp2tZSIm1Ep7tNpEC pPCa+aU3B2fxbkyLlMNGULZxHuXoSwnHqFJpQfR18ryWQAvqXLFRDaAIL wHbbWRAlrySACSvGlojfWWTtuIGtnOaoJiJ6Ev4TwzXPK92BF2jgEX1Pw w==; X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="285821058" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="285821058" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Oct 2022 06:14:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10504"; a="606533150" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="606533150" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga006.jf.intel.com with ESMTP; 18 Oct 2022 06:14:48 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 18 Oct 2022 06:14:47 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 18 Oct 2022 06:14:47 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Tue, 18 Oct 2022 06:14:47 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.174) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Tue, 18 Oct 2022 06:14:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Sa6nBvSr6SVU/s3+UFlrzS+QwCKGt9CAuJtB/Q5lV8NaRTPF9RyBehVcLx2zuD8K6/tdQqSV4du7c0tTanPf6hZQxjDdaoGPmLuDSoI4EuSkNai3q1z2fY1Grb3F6rKHmcgGH0AAS38Nl8I9XHDm4dSYee+pC7si5flUZ7QmwiqRybMCfbso6k9QuvI3qsasZ6vPDY/fk2//xIgxHPV7+RSRgz9MoIQtoF/GpJm5YSe/YSVSiOOTHvIqcyhpVQJ7qiyLxf2YyvpP6nB5bDJIoKUqR75kfQpwnn/oApYoQmkDAgxY8Ou7HOrYl2neb9TCbK9Ut/dtl9slu4DXcaKGMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ze7LKGBIeIsFFWZsySc6QEenAVBvbXMpi7a3+rEJ4ok=; b=BGQIgmrwLosEW899sj4zry3I5kVqMvVrABFg4qAXib6W1jzeygehPTtBIDuQjdKmdR57jdGavVng90Kz8Epl2DLBtkiRcrkwQ1why73u8hTYbzBbF2HFK2jrDjy//sUDNtPYVDcF46SFEs9mvE4lNEYWOl9P8E3hm9txWX0uHJRDc1+PYgPuaVPIq5JqF/Mnb3CUN9unC+d0UAWus4BXducbyZ+jvoQdW5zN15M2KFTrKGywgxHspNpZb5H+QCKAnExvi/GmPwy5Wzu9c0k034m8JMHRN4lPHW7dE/PY1pNZcPEg3KV65WdCBJIxtBomHWUWPG/G28iYJAQhQXLw6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) by PH0PR11MB5595.namprd11.prod.outlook.com (2603:10b6:510:e5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.30; Tue, 18 Oct 2022 13:14:43 +0000 Received: from MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::5582:9796:3aaa:aa1]) by MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::5582:9796:3aaa:aa1%12]) with mapi id 15.20.5723.033; Tue, 18 Oct 2022 13:14:43 +0000 Date: Tue, 18 Oct 2022 14:14:37 +0100 From: Bruce Richardson To: David Marchand CC: , Jerin Jacob , Sunil Kumar Kori , Ciara Power Subject: Re: [PATCH] trace: take live traces via telemetry Message-ID: References: <20221013074928.3062458-1-david.marchand@redhat.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20221013074928.3062458-1-david.marchand@redhat.com> X-ClientProxiedBy: LO4P123CA0408.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:189::17) To MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR11MB1629:EE_|PH0PR11MB5595:EE_ X-MS-Office365-Filtering-Correlation-Id: bfde6f49-2e81-417a-6bb2-08dab10ab8c7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MQluasQIf7337l7oeofTAbr1SK5WD4a/j3qvlKEOxsIx8+nT/+et7LT/lWWp9tQoUmHa5p5FRIepiyT0EWEoHNk2b+jyhVtFF3B9FiErtvUjG7cT9S5j70wrPOXjIViCPFh/r0LQoNXJh0z7UEUZbS2NOtIfeZjliVuOwaZ2RPnmiz1nxuOe5zpujrD3ps1NMLAFbZnddNmCbKLJsfi0Nz9C1Mdthh3MA/ghdiSclA7QOSaNez+kfG2APzP9WKV1A8HmvXNJvHCA1J3efPwP9FrXuzD9UKQkTWA9g/lIlP5kxhJaHlHkErHoeftJl3WmlGW9rC32gX36Rk2dc/BacmwwNfeCbjjAZIGNJB+3Bcuy+dxdlXmLMOGY/N9CmM4tjUCJgZ6OjLywmniNsXWg+jiqGmWPiYgLcOZDxD7yvI+6RT7sT8x+sAtTZAkntOzVKpcR5beSP1RgnS3n2FVJyr8mREUUjs1duNzskFqkwTwXOHYMxf9OE1jKYLVagLf2AF1wgOEF8YQfmhjdaiptPhE8WSVa7nDXBfSDYOnynIPVjP/s7dJGVi96bWHT/zINYZ1Bdlszu0RCRK0LxmqLdfDr0GOrLNfSyWY0LBpgr4YPl8NW3ulGFg1KsaOb6MdIH8HPKp8kcWL1GH9ZmACN4BXGhpq2CZhCebqhH2RaaCm0IUwO2ddimjCG/aYnLjtfU94Ctmscju1WlFjsPDj6Gw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR11MB1629.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(376002)(346002)(39860400002)(396003)(136003)(366004)(451199015)(107886003)(478600001)(6666004)(6506007)(41300700001)(8936002)(4326008)(316002)(54906003)(66946007)(6916009)(66556008)(8676002)(6486002)(66476007)(38100700002)(186003)(86362001)(82960400001)(26005)(83380400001)(6512007)(44832011)(2906002)(5660300002)(4001150100001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pEjTvR9CVD4E0Mm0i9bBC/NEGvaTNBSJW12u3QDs/Elq7Ko1moKAJXnW9WWS?= =?us-ascii?Q?o5NvJIWX/Ml9NEuhSGqoRRC4nEHO5E38xpjpzsiqlLHNenPE9aaYcZt45c0u?= =?us-ascii?Q?JDinmXsrr9xNvJZCLbCvbU7zlHYpKHfh+KVyHqPFhLAhbt802iXyz8hOgRWh?= =?us-ascii?Q?wMAXy/Q/mL+2NmkeCG36/D47FXaF7/OH8RVw+71UJeWvfwRM107hc011wVns?= =?us-ascii?Q?H/3Xi3JFVIc5IeIuvrj7W5SdiWoCnPDb0DbGwsFpjkBsZIUySP4ugvd3EmlS?= =?us-ascii?Q?F3L6bWDGLtASQkIJQ/l8gVulYO5Zgk9Ne9JTDuW5D/2afWLe4gitg8Afhgzj?= =?us-ascii?Q?HBtl7QSufSm90UOJHXJbkY5jtiiIrXZcdscXKuxk46pXl0/3RT3BV/+sbAp0?= =?us-ascii?Q?2HT6qFV4ca+TPzH08SunwNX0CMEXBTMm0NgSvtgw74dbQZI9F69yl4xE1owL?= =?us-ascii?Q?M8FxivAzcGyxktCShu2Os5j35Hrd8/SO2KzxN41rveP7xT2XhyCg2ZQguwkv?= =?us-ascii?Q?+F3lGyecsWZKJ3fr36KRBCAkVNG7KzJ8fQH7IBRDG5MYacOmHbLn0sAor7pB?= =?us-ascii?Q?rsqTWD4hGCrI2vazfpddPqYDcz/I/bai7C/wLgV/mIO43Zf9LEiMzN0bdVl8?= =?us-ascii?Q?RBWHTNssPvXnGljaHyLgj9wozlwfLL2MRP2JobMuAt8uAHWFV/fMB1hhFlyn?= =?us-ascii?Q?invelIjKjr+oMj/T3gA3IikjmhhnR6qv/u7Y+GkyBhTZ+Klxmjupzy5PQjkT?= =?us-ascii?Q?3lUjEYBwP8M/5T4j1hYmtsAi2ee915RoMu0SCOanGdkmEojGFfdURdZLM43h?= =?us-ascii?Q?ShBbPp4r7pK1fzoluSZ5HDQO28W/RRrPH4tuNcN0Jpov4N43wfQAJ5HZ+x6E?= =?us-ascii?Q?IodAn2cJk0hN2hpN+tNe5M8Zmr+aRTJM3M5I4RWtw5YcRDZ+NEqSgPWbioNC?= =?us-ascii?Q?xOoRoBZi3kSk9Mlf5El1I/5IXCPkPnBz7F6+umfcfG2+uxnpn3Y1p5rS8o/5?= =?us-ascii?Q?5aDvU0oxYYxlKmedIFQiW4UaCDo2FJWqOdCA8MljXh1qeDdJv0/+Lvvr0Q8Z?= =?us-ascii?Q?7qhLK/UFgTNrPdlug9TxN8jEX2q8uWYySOvsuhSRuSjQtNSuQWZZr24eFuHs?= =?us-ascii?Q?2fmG/tWFXQYpnCCbAu15Rrcwi0X3ZkepXoYXJWeFeYmz0HTVAqWmKBWN/fY6?= =?us-ascii?Q?Jjud3LC1WMm77Pn8m6smrGWyh7GwaW1X+drU/B43nJv76qR6SMiR7SZF8OtB?= =?us-ascii?Q?ZTAxofJUn+UXxAa84HmCmTnJD05DKV+bheCfkJU0FK8faOJhxnvMeG+fR4Wo?= =?us-ascii?Q?la8iBYOhGO8Cax2Mnu4yk3pqE0GjuDAY9sLQNenfNJ95ali8xH83bVaYeqJ4?= =?us-ascii?Q?wLAkWcj4k7IkrBr1DkbTHnCrIc6SpNNbyyiqyntdtHf/ir/y0/pA6k9OvQr/?= =?us-ascii?Q?mmsn0x/gU7tJGSn6Mo0mE2eL/sT5lNbYcyH6Wonny8QP9cy3LBUdvixKbED3?= =?us-ascii?Q?ku4nGqvMQwYOCuHbUsi9BwaV+sq9nEpXs2cE6CmdqzLfSSSXwI28CQmoY5Qb?= =?us-ascii?Q?a8V8HKbCi+O/tj2+j/Lp6pN8MVczE0ehFPas0J1EjJxXOe7XZOP+K9FyUr//?= =?us-ascii?Q?+w=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: bfde6f49-2e81-417a-6bb2-08dab10ab8c7 X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1629.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2022 13:14:43.6869 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oIf0lQje+m6/0tf0aq9NaJeRX+GGlGKorg2//HB1d4gICVUgayzTpdy+Nn1k5QUn++S4+mHMVIatJzx2EvuZdFc+HpGHbb1OVhp9idsz4WQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5595 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Oct 13, 2022 at 09:49:28AM +0200, David Marchand wrote: > Register telemetry commands to list and configure trace points and later > save traces for a running DPDK application. > > Note: trace point names contain a '.', so the list of valid characters > used in telemetry commands and dictionary keys has been extended. > > Example with testpmd running with two net/null ports (startup command > from devtools/test-null.sh): > > --> /trace/list,lib.ethdev.* > {"/trace/list": {"lib.ethdev.configure": "Disabled", > "lib.ethdev.rxq.setup": "Disabled", > "lib.ethdev.txq.setup": "Disabled", > "lib.ethdev.start": "Disabled", > "lib.ethdev.stop": "Disabled", > "lib.ethdev.close": "Disabled", > "lib.ethdev.rx.burst": "Disabled", > "lib.ethdev.tx.burst": "Disabled"}} > > --> /trace/enable,lib.ethdev.st* > {"/trace/enable": {"Count": 2}} > --> /trace/enable,lib.ethdev.st* > {"/trace/enable": {"Count": 0}} > > --> /trace/list,lib.ethdev.* > {"/trace/list": {"lib.ethdev.configure": "Disabled", > "lib.ethdev.rxq.setup": "Disabled", > "lib.ethdev.txq.setup": "Disabled", > "lib.ethdev.start": "Enabled", > "lib.ethdev.stop": "Enabled", > "lib.ethdev.close": "Disabled", > "lib.ethdev.rx.burst": "Disabled", > "lib.ethdev.tx.burst": "Disabled"}} > > testpmd> stop > ... > testpmd> port stop all > ... > testpmd> port start all > ... > testpmd> start > ... > > --> /trace/save > {"/trace/save": {"Status": "OK", > "Path": ".../dpdk-traces/rte-2022-10-12-AM-10-51-48"}} > > $ babeltrace .../dpdk-traces/rte-2022-10-12-AM-10-51-48 > [10:51:36.229878723] (+?.?????????) lib.ethdev.stop: > { cpu_id = 0x0, name = "dpdk-testpmd" }, { port_id = 0x0, ret = 0 } > [10:51:36.229880251] (+0.000001528) lib.ethdev.stop: > { cpu_id = 0x0, name = "dpdk-testpmd" }, { port_id = 0x1, ret = 0 } > [10:51:40.449359774] (+4.219479523) lib.ethdev.start: > { cpu_id = 0x0, name = "dpdk-testpmd" }, { port_id = 0x0 } > [10:51:40.449377877] (+0.000018103) lib.ethdev.start: > { cpu_id = 0x0, name = "dpdk-testpmd" }, { port_id = 0x1 } > > --> /trace/disable,* > {"/trace/disable": {"Count": 2}} > > Signed-off-by: David Marchand > --- > For runtime testing, please use this patch in addition with series 25183. > Depends-on: series-25183 ("Trace subsystem fixes") > > --- > lib/eal/common/eal_common_trace.c | 78 +++++++++++++++++++++++++++++++ > lib/telemetry/telemetry_data.c | 1 + > 2 files changed, 79 insertions(+) > > diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c > index f9b187d15f..9a54987b42 100644 > --- a/lib/eal/common/eal_common_trace.c > +++ b/lib/eal/common/eal_common_trace.c > @@ -12,6 +12,7 @@ > #include > #include > #include > +#include > > #include "eal_trace.h" > > @@ -530,3 +531,80 @@ __rte_trace_point_register(rte_trace_point_t *handle, const char *name, > > return -rte_errno; > } > + > +static int > +trace_telemetry_enable(const char *cmd __rte_unused, > + const char *params, struct rte_tel_data *d) > +{ > + unsigned int count; > + > + if (params == NULL || strlen(params) == 0) > + return -1; > + rte_tel_data_start_dict(d); > + count = __atomic_load_n(&trace.status, __ATOMIC_RELAXED); > + rte_trace_pattern(params, true); > + rte_tel_data_add_dict_int(d, "Count", > + __atomic_load_n(&trace.status, __ATOMIC_RELAXED) - count); > + return 0; > + > +} > + > +static int > +trace_telemetry_disable(const char *cmd __rte_unused, > + const char *params, struct rte_tel_data *d) > +{ > + unsigned int count; > + > + if (params == NULL || strlen(params) == 0) > + return -1; > + rte_tel_data_start_dict(d); > + count = __atomic_load_n(&trace.status, __ATOMIC_RELAXED); > + rte_trace_pattern(params, false); > + rte_tel_data_add_dict_int(d, "Count", > + count - __atomic_load_n(&trace.status, __ATOMIC_RELAXED)); > + return 0; > + > +} > + > +static int > +trace_telemetry_list(const char *cmd __rte_unused, > + const char *params, struct rte_tel_data *d) > +{ > + struct trace_point *tp; > + > + rte_tel_data_start_dict(d); > + STAILQ_FOREACH(tp, &tp_list, next) { > + if (params != NULL && fnmatch(params, tp->name, 0) != 0) > + continue; > + > + rte_tel_data_add_dict_string(d, tp->name, > + rte_trace_point_is_enabled(tp->handle) ? "Enabled" : "Disabled"); > + } > + > + return 0; > +} > + > +static int > +trace_telemetry_save(const char *cmd __rte_unused, > + const char *params __rte_unused, struct rte_tel_data *d) > +{ > + struct trace *trace = trace_obj_get(); > + > + rte_tel_data_start_dict(d); > + rte_tel_data_add_dict_string(d, "Status", rte_trace_save() == 0 ? "OK" : "KO"); > + rte_tel_data_add_dict_string(d, "Path", trace->dir); > + > + return 0; > +} > + > +RTE_INIT(trace_telemetry) > +{ > + rte_telemetry_register_cmd("/trace/enable", trace_telemetry_enable, > + "Enable trace points matching the provided pattern. Parameters: string pattern."); > + rte_telemetry_register_cmd("/trace/disable", trace_telemetry_disable, > + "Disable trace points matching the provided pattern. Parameters: string pattern."); > + rte_telemetry_register_cmd("/trace/list", trace_telemetry_list, > + "List trace points. Parameters: string pattern."); > + rte_telemetry_register_cmd("/trace/save", trace_telemetry_save, > + "Save current traces. Takes no parameter."); > +} > diff --git a/lib/telemetry/telemetry_data.c b/lib/telemetry/telemetry_data.c > index 34366ecee3..5b319c18fb 100644 > --- a/lib/telemetry/telemetry_data.c > +++ b/lib/telemetry/telemetry_data.c > @@ -106,6 +106,7 @@ valid_name(const char *name) > ['a' ... 'z'] = 1, > ['_'] = 1, > ['/'] = 1, > + ['.'] = 1, > }; > while (*name != '\0') { > if ((size_t)*name >= RTE_DIM(allowed) || allowed[(int)*name] == 0) I don't see an issue with allowing "." characters in dictionary names, so for this part: Acked-by: Bruce Richardson