From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA055A059F; Fri, 10 Apr 2020 16:59:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 67B891D5B0; Fri, 10 Apr 2020 16:59:40 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id D3F021D54D for ; Fri, 10 Apr 2020 16:59:38 +0200 (CEST) IronPort-SDR: zSFryRyOpvDAQuD66nDntZitAxqlaFFhxIXCiATkuZupUWvYMl/Lhr0gfANgD4Y95acf3OdV27 yrQ+uIvMGIxw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2020 07:59:37 -0700 IronPort-SDR: h0edeDOXoIlCA8v5eR0qpycaO1dEjNy15aw+Y2iC6N9Gq9zCRu40+XY60AIBVkdsJ1CYoQInoa NhjdqywSwWUg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,367,1580803200"; d="scan'208";a="297820922" Received: from orsmsx105.amr.corp.intel.com ([10.22.225.132]) by FMSMGA003.fm.intel.com with ESMTP; 10 Apr 2020 07:59:37 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX105.amr.corp.intel.com (10.22.225.132) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Apr 2020 07:59:37 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.104) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Apr 2020 07:59:36 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Us+88X9hLnyTWDaBbL2e6oncYKyC1tErLih5lKoPWGziXctdP4bZAUnM17M82dtamAVvTHOtL+QP+ZuQy5Auroy6exJ4aCV6fTvrsnARYGVd3mytp4qjX7vpklPA5w77Pnj5Kb8KhmDBXx/CqNIW90wiDV0mNxBQkGts+DtwA+G7o3QqMAchpdkRRlRaZ4PY1aPTZPrRdt69yCM6P7agrI/Dqkrc12K143LH2GWXNSa9L0TsjcvslJXgFtaLdQHgzmXEQXfYF2cekcdasbyTc560KAPAufV2y2fFVrdvNOexDk1TM85SDgO12I47yCDx1JXlDua+4pU5ht1pnTVJ+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MdgvzaOrMwE7L6+xqcPEy2f8GA81hXNotZw8mzzrRgs=; b=Jp/OjU7pyd69c7GAxE3U6wGgvTBo98NG79AoQopa9xBlQ+mxLdRrqTIu6JpA9ai5TcRQPuXK5qIvmHW5K2pFBP7V8gao2/Yx6dOgkyeG1rl6Xyi5BbmrC08VkfcZGMuehPO95ki6sj49Ut634ZVhDNaXcjFVU3jk+Q3yvmkD4b4gDcnvQNQm+2mAuojNSvvuV/r7imiNlCsi0UQhdbnIEBhodQXhXyQQlkqUS0AlEJSUPXcXz/C8d9ArLNTnk0nwUlCLrXvcAsYXC/XmWJC+r1J2YtejSvt5yRq3IkkGMWRBKeRM0FqJTBgw3L22jB7AkyXunoueCC1DGYuZrUojrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MdgvzaOrMwE7L6+xqcPEy2f8GA81hXNotZw8mzzrRgs=; b=sxwiL7T9hngUr8f97eXKJG8UwZmlqbq1lrFWaQlQMkVBmlKdzlpps7A4Wh1ooEAuocEXIo3bf0JTW2Pyqolq5tPcAsTRCH8qYeAQECoprhEcSS2wgI8bHtymXYIetczAqjTQu+czcAmloTN91QFMwCikPSeT9BifBFwyitUKNIo= Received: from DM6PR11MB4593.namprd11.prod.outlook.com (2603:10b6:5:2a3::8) by DM6PR11MB3385.namprd11.prod.outlook.com (2603:10b6:5:c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.17; Fri, 10 Apr 2020 14:59:33 +0000 Received: from DM6PR11MB4593.namprd11.prod.outlook.com ([fe80::9d18:a682:243f:6d6d]) by DM6PR11MB4593.namprd11.prod.outlook.com ([fe80::9d18:a682:243f:6d6d%3]) with mapi id 15.20.2900.015; Fri, 10 Apr 2020 14:59:33 +0000 From: "Wiles, Keith" To: Thomas Monjalon CC: "Richardson, Bruce" , "Power, Ciara" , dev , "Laatz, Kevin" , "Pattan, Reshma" , "jerinjacobk@gmail.com" , "david.marchand@redhat.com" , "mb@smartsharesystems.com" , Andrzej Ostruszka Thread-Topic: [PATCH v2 00/16] update and simplify telemetry library. Thread-Index: AQHWDchjQaQiU90Sg0y9/LABPPMO4qhvhG8AgAD/2YCAAAU4AIAB5nyAgAADXICAAAJfAA== Date: Fri, 10 Apr 2020 14:59:33 +0000 Message-ID: References: <20200319171907.60891-1-ciara.power@intel.com> <8682539.rMLUfLXkoz@thomas> <93132D13-8044-43A7-A817-4B526BEEBC29@intel.com> <12264742.EVyyLHbfrO@thomas> In-Reply-To: <12264742.EVyyLHbfrO@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=keith.wiles@intel.com; x-originating-ip: [192.55.52.207] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7a287068-a865-407b-183a-08d7dd5fc799 x-ms-traffictypediagnostic: DM6PR11MB3385: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0369E8196C x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4593.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(396003)(39860400002)(346002)(376002)(136003)(366004)(6916009)(478600001)(36756003)(71200400001)(86362001)(2906002)(186003)(26005)(966005)(316002)(53546011)(2616005)(54906003)(33656002)(6506007)(4326008)(91956017)(6486002)(76116006)(66946007)(8936002)(66476007)(5660300002)(8676002)(6512007)(64756008)(66556008)(66446008)(81156014); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IMhWwhQKpgcq++hAteekXoI3Anohv5NmbYNYiVYVBlf1CFXaZpa5yWT4z+Jyx/IvZPFX31OoK0hMdv4HDWRkm1mQ79MjDO7jBjYoJF2sWL8lGYYV8fok3o5hrcka+o0Qiniz8iwk5C4cEwIEVim2hHcUiY0penb5LTvOnTXx0bCXfoJ9eZoUrmqF0yglJ0/jDUzSmhHO/y8F+Pi8M55tQ3WUz2n6j1xF6fJG7uMYiGUBuf+/iNnFv7LyUN1IwC8ndMYEVHm/i7TfU4v2zNicGfltij9TGnGTn+x33jHjMdshrPSdntcnzSes9BuVfNjF4qEYYUpQxoeSZPaa7Q3f/WGCPXy3yqok85l9mRXwuRQm1FzvSCF7Yj+174glK35tAgSGEnCab1ZYGGH/NZ1aY6EEO7zoePukahH05EkIXEiErIqj03/j1BuvQnSQYBOCVZ6JVPNqL7QOe4UV0kmBh5uOHswOg/PeMX+Lx8uYaIVfeCiqCjGF1ydq7BWeBBEfD2WZn2m3p5hNzlLxwUADWQ== x-ms-exchange-antispam-messagedata: ndrWWA7zZ5sNHo07n1t1N1b3OlL/kySqdFNSV8akutkR+aktfC4hA8JsR71Y5ltL7nsSI6HojCRL4kYz2LmAvo6thkj9L4Fym5iQWvMkyc6DEy+l2s3nRIN64mq+8qYPH2S//0C1Wz8Wgr8+5a1VQg== Content-Type: text/plain; charset="us-ascii" Content-ID: <474C1348066BD045AB1ED9F64FCFF690@namprd11.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 7a287068-a865-407b-183a-08d7dd5fc799 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2020 14:59:33.4972 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ZBEZzcNOE2bjBmasIjnTNRtSxXVcHOwluDGIQQlJWAOzS/dWYTeGYTv8EYO1963LRLjlc5RzJFrRWyfgoeltuA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB3385 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > On Apr 10, 2020, at 9:51 AM, Thomas Monjalon wrote: >=20 > 10/04/2020 16:39, Wiles, Keith: >>> On Apr 9, 2020, at 4:37 AM, Thomas Monjalon wrote= : >>> 09/04/2020 11:19, Bruce Richardson: >>>> On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: >>>>> 08/04/2020 18:49, Ciara Power: >>>>>> This patchset extensively reworks the telemetry library adding new >>>>>> functionality and simplifying much of the existing code, while >>>>>> maintaining backward compatibility. >>>>>>=20 >>>>>> This work is based on the previously sent RFC for a "process info" >>>>>> library: https://patchwork.dpdk.org/project/dpdk/list/?series=3D7741 >>>>>> However, rather than creating a new library, this patchset takes >>>>>> that work and merges it into the existing telemetry library, as >>>>>> mentioned above. >>>>>>=20 >>>>>> The telemetry library as shipped in 19.11 is based upon the metrics >>>>>> library and outputs all statistics based on that as a source. Howeve= r, >>>>>> this limits the telemetry output to only port-level statistics >>>>>> information, rather than allowing it to be used as a general scheme = for >>>>>> telemetry information across all DPDK libraries. >>>>>>=20 >>>>>> With this patchset applied, rather than the telemetry library being >>>>>> responsible for pulling ethdev stats and pushing them into the metri= cs >>>>>> library for retrieval later, each library e.g. ethdev, rawdev, and e= ven >>>>>> the metrics library itself (for backwards compatiblity) now handle t= heir >>>>>> own stats. Any library or app can register a callback function with >>>>>> telemetry, which will be called if requested by the client connected= via >>>>>> the telemetry socket. The callback function in the library/app then >>>>>> formats its stats, or other data, into a JSON string, and returns it= to >>>>>> telemetry to be sent to the client. >>>>>=20 >>>>> I think this is a global need in DPDK, and it is usually called RPC, >>>>> IPC or control messaging. >>>>> We had a similar need for multi-process communication, thus rte_mp IP= C. >>>>> We also need a control channel for user configuration applications. >>>>> We also need to control some features like logging or tracing. >>>>>=20 >>>>> In my opinion, it is time to introduce a general control channel in D= PDK. >>>>> The application must be in the loop of the control mechanism. >>>>> Making such channel standard will ease application adoption. >>>>>=20 >>>>> Please read some comments here: >>>>> http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ >>>>>=20 >>>> Hi Thomas, >>>>=20 >>>> I agree that having a single control mechanism or messaging mechanism = in >>>> DPDK would be nice to have. However, I don't believe the plans for suc= h a >>>> scheme should impact this patchset right now as the idea of a common >>>> channel was only first mooted about a week ago, and while there has be= en >>>> some email discussion about it, there is as yet no requirements list t= hat >>>> I've seen, nobody actually doing coding work on it, no rfc and most >>>> importantly no timeline for creating and merging such into DPDK. >>>=20 >>> Yes, this is a new idea. >>> Throwing the idea in this "telemetry" thread and in "IF proxy" thread >>> is the first step before starting a dedicated thread to design >>> a generic mechanism. >>>=20 >>>> At present though, DPDK has a telemetry solution that works for the us= e case >>>> of ethdev stats and some power management info, but requires a more ge= neral >>>> solution to allow monitoring tools like PMDT to introspect DPDK, and a= lso >>>> to prove statistics for other parts of DPDK such as cryptodev, eventde= v, >>>> and other libraries, plus the application itself if the app so desires= . >>>=20 >>> Doing rework on telemetry is similar to a general control mechanism. >>> Can we take this opportunity to work on what we believe to be a bigger >>> idea? It should be done anyway, so why pushing this temporary solution? >>> Sometimes we need a quick answer to an urgent problem. >>> But I don't think telemetry is currently in such situation that >>> a rework in 20.05 is mandatory. >>=20 >> Updating telemetry to be more consumable and standardize on a single met= hod to get stats/info out of DPDK is a clean and simple solution. Starting = over and creating yet another solution means we are pushing this support ou= t again and many customer are asking for this support now. >>=20 >> The current telemetry solution in this patch gives us a great starting p= oint and going back to the drawing board is a waste of time IMO and we need= something now. To me this is a urgent problem we need to solve now, as I w= ant to push PMDT and if we keep pushing out this type of support then it wi= ll never be upstreamed. >>=20 >> In PMDT I believe I have resolved all of the tech boards concerns and ju= st waiting for this patch and a patch to PCM to push the code back to DPDK = again. >>=20 >> So please let's not redesign this again. >=20 > I understand your concern. >=20 > I think we need to go to the drawing board, > and consider at least these 5 use cases: > 1/ multi-process IPC > 2/ telemetry > 3/ IF proxy > 4/ external user configuration > 5/ log/trace start/stop >=20 > Merging telemetry means we'll rework 1 and 2 later. > I am OK with merging telemetry in 20.05 if we can be sure > that there will be no resistance and help for reworking it > with a more general communication channel if required later. >=20 > We need a kind of community vote here. Please give +1 / -1. > Giving +1 means you will help when needed. >=20 I agree merging the telemetry patch for 20.05 is reasonable and addressing = the above concerns later is a good compromise. +1 for helping on the above concerns later after the 20.05 release and merg= e of telemetry patch. >=20