From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2EB2B41E86; Mon, 13 Mar 2023 10:39:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14EA4406BC; Mon, 13 Mar 2023 10:39:16 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id DDA6140151; Mon, 13 Mar 2023 10:39:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678700354; x=1710236354; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=YUpCW24FVcDtshTSHsjh+BE+MTY6AscQKcLn9wxE/GA=; b=faxb80pV8Oi4RW9UQE72Hro+ljl6LFoiFep9dAa3U7nGjpvqLh6kdYbq z6zhU0ygCrC+fOfyi16eTtVDQon3Rys507u+4eeA6Q5Kgi+3WOB0RapnV FGBRi+HlXV9Y2NYxITbvr845ot1Z7ivDMHBQkuPONtPRnMzXjz0Yvo4q2 R2sOt1/TzHAbVpNb0w61ofmQgMutn2fzcDFqQEEIxRHME6+oZ44mQcPL9 4IBu48xpua7io8whFboa8xPSR77rBl6WCJOjIs5u1FaqnuUZvHI6hNtmK Sj+J6NuQzX6Hb3QVMGZbGLaOCk9TL+8iOANyKkfytFgd86Iv0TKPy6M8l Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10647"; a="325461417" X-IronPort-AV: E=Sophos;i="5.98,256,1673942400"; d="scan'208";a="325461417" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 02:38:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10647"; a="742793759" X-IronPort-AV: E=Sophos;i="5.98,256,1673942400"; d="scan'208";a="742793759" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga008.fm.intel.com with ESMTP; 13 Mar 2023 02:38:55 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Mon, 13 Mar 2023 02:38:55 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21 via Frontend Transport; Mon, 13 Mar 2023 02:38:55 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.45) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.21; Mon, 13 Mar 2023 02:38:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DemZohE4RmM+15+EpyrA++gtauLVXrDr/yW084wG+l8rMQSuB68FCGwV61w9U4ypTFJOW0epj6fcm8BjWlEr+/WJoLH65gtSPHzPGjnZyPzEKce6WEV0/54KRhbYMlvQXuZ4ScsHLTbYZPe/zV24vEAcZ3PEUFJpikYkgAHniXuZFYY/uKvCbhhtGUM1raQp4igg1TFZ7+Q55Ky7uMxyMIXcb5Spx26iuHUaWIK+V3LRwq6G9ORRl+hYWIF+bvpqS8Xg/3uX6vd1e1p9osOy7JMMu76+frjbirB9q3/aMDx9rTc40bYu63vI1p6g2EN6vRQuOZSo5pJrPp82sg8xyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dMqQZwZlCb1SZLf4PJHlK3nhGYBYxAkht8gr525J2Bw=; b=GTUG83DntISJ3Iy04oBlEelqzKvsHfajRkq7Zv9kbW0wfBZY+a0f/GSLMvHKfgRVN0WJcKgMIaTuypwdh5C69HRovucgLhPmHidl+N9XrIz2OsdVexO/VDd8ZHRa8hgRtk7v3so3QxfO4kWXjKghBO8qkvp3spyGKDz+6LTiDFs0f5IOHMI6LFYXHqRYPMJ5vzxcGkaSHIv0FKKs0zV6mtJy/jTGJqU0X7i4Tcg/vJd7XjU0PGW6gZaiQB1StDdqVI8oJguin++Eq9RsPQtYJXFdUi1xGjHcVaGNBQ6stieH05KFKQXS4z1udzRYi3Ohas7GkhBW3fxJOtFRJ8l/JA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by MW4PR11MB7078.namprd11.prod.outlook.com (2603:10b6:303:219::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.22; Mon, 13 Mar 2023 09:38:47 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::18d0:ac53:aa1d:d19c]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::18d0:ac53:aa1d:d19c%5]) with mapi id 15.20.6178.024; Mon, 13 Mar 2023 09:38:47 +0000 Date: Mon, 13 Mar 2023 09:38:41 +0000 From: Bruce Richardson To: Stephen Hemminger CC: , , Ciara Power , Declan Doherty , Radu Nicolau Subject: Re: [PATCH] telemetry: fix autotest failures on Alpine Message-ID: References: <20230310181836.162336-1-bruce.richardson@intel.com> <20230310110832.1cfa47e5@hermes.local> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20230310110832.1cfa47e5@hermes.local> X-ClientProxiedBy: LO3P265CA0004.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:bb::9) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|MW4PR11MB7078:EE_ X-MS-Office365-Filtering-Correlation-Id: 5671cf11-f70f-4141-e335-08db23a6beb6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s0o5UMcYTwfu3QFRdMsOtvEcVNXh3WFiFQ/50y6zhYIBU5PitylzbNtkOVicNEDoIbG6mAd4Zfyv/3QwOazunCNODS1AUide3HM9W6aDHuccXcj5YaLBN0kSqAMIlMZLLN8SSHuiBJpsjdlN29XesOCxpNSKB2YUlWQc1uIBWLYgHFembAV46PSX0mg+xJQn97poHJpEFbioEXMCaU3zSW2prFSN1fhh1aThJ2IK1cOEYBXQkcq9t/5REo3DWUpy3T5GMxffwlo7NaaSOywV3axfOKvkkeyuysY/HggtBUya45JAgO+713Oyl5JelBAXX4iG+ya99Res1nWNjjs3ZYJtp3veqAwWURgjj9T1KqF+xqgzM5ZWVR/0sFLvmAKzWt1YBNcP5Z7qnOZTiYGOxycTo3u1y2otZCNnnewjfn+QsAu297sY2s7PhfNEFzj7wD+u9OWFheIUOwZPaheM5BbZgtHsucx0FXYmqaUDkg+9ojUOnv+R0DCuUpyyjr81jbre+hPjGNIr5rzVYbO104ZwUc8PeXfV8BporaFYUPfAte5SgwJ3fqHCXzU/+drcikSOvFM/QhyMpLE2ojDpE2q8mU7TzZST63oXT94N+647UIfwufiYPBp2kxT393vafDLfZvzOw6Nbf9Nt8vRH/jPwi4GOlwB9kh6oKBhKnkBeKkICkl2PaBpNp3UeXOma X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(39860400002)(346002)(366004)(376002)(136003)(396003)(451199018)(82960400001)(83380400001)(6666004)(54906003)(478600001)(38100700002)(316002)(6486002)(966005)(107886003)(26005)(6512007)(6506007)(5660300002)(186003)(41300700001)(2906002)(44832011)(4326008)(8936002)(6916009)(66476007)(66556008)(8676002)(86362001)(66946007)(21314003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?yAWYrWe8fXWLBd5aau6JkIuwbK55m3kbCFJPfWC+Cbm07RbMoAI1puJxjS3N?= =?us-ascii?Q?Pvn5B9CnSnWww9qrQ/HuPEyvKaIgzEHwDYBpQwuWxMuMSFV/sPiaLWsImNmP?= =?us-ascii?Q?vF1EJqJFBcjn1QpcTBFwi7dgc6AitBg4yPvs7yumLLckz5FaueAgTurK8+k4?= =?us-ascii?Q?PYdMWyTR40IxyacTyPXt02jGnUfQU+gBeTDzPTkhS1IAnWr79aJzPzUEMVQZ?= =?us-ascii?Q?f8ylH/cx5B7KpwrlbX02LCQRbrCkKyiO8BvUPyR4ME2SnMVPtrIPpFA9GCYy?= =?us-ascii?Q?xliAn80jARYr0hjdJ1HxE8eUHKbsgDW2evJAzM6Ox4SQtN+3M44RzygC3mRD?= =?us-ascii?Q?4GHAXuHV1atzdpS/VGZNL9QNvwclUrTlZ9AUUk7Qa4pMbXOOiGZvArqeYwT6?= =?us-ascii?Q?zQlswlAy9A5qXAZx4wMeAbSc7cZYd7PO3+RLqu4mfy7E3RIp++xipWo8VYZF?= =?us-ascii?Q?dhUG9unJ6xDnSEdxlNtimlQRpj5NA2/5soD1CPzavVeVmfAnthwHZ9LPu01Q?= =?us-ascii?Q?yFtTbqHCaZXEVFY4hdBO0jtc6hzo9j1vZsm/bW++iCdA/6IJWyxSPoLT2Qgb?= =?us-ascii?Q?dMGcBNrLClvJeZJKIIq646RGYfZizQb4hQTOptVKE5RpAt0smZQGd/Y5dLBT?= =?us-ascii?Q?/S5xsXI5Lhqzr5stdHlrpMV1Hl++eNz9uPnIZxsY2SfhGhylrFiHsvim5Vz5?= =?us-ascii?Q?mF8U7DRRQ2BAV+r4GKONSrk1i21agVZcxtUoO15c9IjkX78ywOo0EeD58ICV?= =?us-ascii?Q?1SsBtE2YzQKBzN2+v3PYeDj8Ocw9rP/zzTGkWJkK8eY87TsKh+F+KqcWmjZ9?= =?us-ascii?Q?rbC1YCZrZZ6GLVnMpQcNnmI9IzHNr3CzZ5XJMe1UTnDCMr9TzLEzmJJGLwWe?= =?us-ascii?Q?suSM8/wgFbsRbJ4MulJusTv6352DM9Rehlq1skrTECjMSdZX2UEp8wR2BT4d?= =?us-ascii?Q?8MLrCljqq8KH2r5m/mCcvUqLxfxBorhRCgTbZTkCDrWGudE+9GhZUm2ulJWD?= =?us-ascii?Q?p1TsvwmzW0vg+Y3BkSEMMUfuVTv02tOwGzBzwUTRL7LQW0RJDOCmbLIXmB7x?= =?us-ascii?Q?l8nfNz1aLGDsca903VFzymb3R6eirhKnhu/Pm4oq/i4jxjJLAd3VXxeBFNzi?= =?us-ascii?Q?z9wGxj8xuYYuXe8ZOpNaNk3zW7iTqT+6/vTsM6iyLKESiqKx6vOIdRTVcbt+?= =?us-ascii?Q?70ufrlb5OjMiFhh82QZNumcP2TLGXLLKfV8N+h7L/A7kZIy0E3bonU5W94Qw?= =?us-ascii?Q?JK1/9IuYogAT14SXUdmMOqcHMeePn8V/NgA99JSzVnLJ5mDv7JCQCZ46ujW3?= =?us-ascii?Q?LMRRlCfr31BKdAsLLxp4ano8TygRKGPSf9Sxl/SHlpSOtklMyqE9OeCWRc6/?= =?us-ascii?Q?nU7cYCtudE4ZorTTITcREs10scBBvFKUd2R1z9rSRgc0rOkBG5CbPVug8ts5?= =?us-ascii?Q?raAeti+EWTDbmWbt0aV3leWEaCZZxtrJXrm59T34eP14dCqDYnat7E7zSsQU?= =?us-ascii?Q?zYn42+sYZ+tiS5X12akgRoPEDTVfMztS3d4i1KffmhIsWB5PXuHKqkLC4UWj?= =?us-ascii?Q?hu4GqLhO5RgIZYMQrwknaeIWRmWa/r8LdyeBlUh1jxP1Mx8PL0Fbe+O3gFRJ?= =?us-ascii?Q?4A=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 5671cf11-f70f-4141-e335-08db23a6beb6 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2023 09:38:47.6132 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aWh5o0BbSKODRMIOtiiIyKZVJj8Psg7R6uljsw4+6U2yNHLTuD7Ej5uO5gW3/3YLor1kxRMuSVwzR8/e5aQnbPbxFvhiVid6Foi5faqsUZ8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB7078 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Mar 10, 2023 at 11:08:32AM -0800, Stephen Hemminger wrote: > On Fri, 10 Mar 2023 18:18:36 +0000 > Bruce Richardson wrote: > > > On Alpine linux, the telemetry_data_autotest was failing for the > > test where we had dictionaries embedded in other dictionaries up > > to three levels deep. Indications are that this issue is due to > > excess data being stored on the stack, so replace stack-allocated > > buffer data with dynamically allocated data in the case where we > > are doing recursive processing of telemetry data structures into > > json. > > > > Bugzilla ID: 1177 > > Fixes: c933bb5177ca ("telemetry: support array values in data object") > > Fixes: d2671e642a8e ("telemetry: support dict of dicts") > > Cc: stable@dpdk.org > > Looking at the telemetry code: > > - why so many temporary buffers, could this be streamed or redesigned > so that an allocated buffer is returned. > This is largely what the fix patch does. The temporary buffers are used to ensure we never end up with a partially written buffer from any truncated snprintf, since that could cause us to emit invalid json. I think as a scheme it works well enough for what it was designed for, but this particular test case has more levels of recursion than was expected, so the limit of the design are showing. The downside of using memory allocation using malloc as here, is just more failure cases. > - why is rte_tel_json_XXX all inline? These should just be internal > functions and not in a .h file. > Sure. Since these are all just internal functions used by the library, I'm not sure it really matters. > FYI - if this library reused existing json writer it would have > been much simpler. > https://github.com/shemminger/iproute2/blob/main/lib/json_writer.c Yep. Looked at that previously, but it's at a lower level than what we have. IMHO, the complicated bit of producing json output is not the formatting characters for each data-type but ensuring correct termination of the json string even in case of errors, so each function in our telemetry json code always includes correct terminators at all points, i.e. we have no separate functions to be called for terminating objects, arrays etc. - they are always included in the output of the items. This is why so many temporary buffers are used - the input string to a function is well-formed json, and we only actually append to that buffer by copying from the temporary buffer once we are sure that the output will also be similarly well-formed. That said, if you want to replace the current json implementation with a better one, I'm not opposed to it. [though I'd definitely rather no external dependencies which would mean we could no longer rely on it always being available in default builds] /Bruce