From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD5ADA059F; Fri, 10 Apr 2020 16:39:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 936CE1D546; Fri, 10 Apr 2020 16:39:08 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 47E731D546 for ; Fri, 10 Apr 2020 16:39:06 +0200 (CEST) IronPort-SDR: SQVLyw5OH5ojX5C+YsBhUIEUd3ecQ+eMiwt3EvimRkF6AxHZYZzNLy6eKZZQC6/fFuZ6AWaJSF eRgYjsRw6xHQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2020 07:39:05 -0700 IronPort-SDR: 8PJOhnbQcFONguhGjWu13Kw0DNHNRLqGmw+oSgyvF2IWYL2XaNK57tA7t+3SqVWn9lgih2pELy 4NWN6Tas9tVg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,367,1580803200"; d="scan'208";a="362456058" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga001.fm.intel.com with ESMTP; 10 Apr 2020 07:39:04 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Apr 2020 07:39:05 -0700 Received: from fmsmsx606.amr.corp.intel.com (10.18.126.86) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 10 Apr 2020 07:39:04 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Fri, 10 Apr 2020 07:39:04 -0700 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (104.47.44.51) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 10 Apr 2020 07:39:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aHlOkEdfJfOjdD0+y8iw+hlrfBPubizOUAZYM3ux/d+1mJ/CHE6DPHV1U1+AT8CF6Ss9CStr9DNjLkTyWxCuWNAiJboTof1PjGCD6eSyxFM4vgaOchwzQHU5PIfBJgcTet0CWeLIsNEe27Nt1lOt58d+Ksm2x+sxVsPaBuCJyQube+aWp/I5GXVyQggKQ+ojtucqRNy5FBzMa+sEydFalk2Q270EpAJdWjRT29wOPndx06/+FnfyJC7FU86FeD9IXqKFwtIgmsTSMh0wT2ORoSud/gnaPGtJLtgyJRBa5PbOoka/IyRaXhEdODmWRIw6UlZzO64UStB4B82ehYDG/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zMN+134y00IOPAGbHxYb1JZP3yvBcsgKEciBTox9s/U=; b=U3ihUMkFw+3d4pTYZaRHFOqicAl6d2ArUHaPAvJOFSlcSbiv2f91JvAiQEPqeMvKq12mhr/0iFLn3or12Rb+BziTXMDsXiCiXDc480qqtxT5dNt7ryKQlYpqyoVBv9GVESlAygLRHuyH72J+0GMr1aS6KyT6wvLzxEVQq2ZaZnguKA16b0wZQ6ixasujET4eWnzQOzkTtOcBtqOA6lfW3BECqgomP9d8pvOMUERrGpk/YTdF1zZgUjEW/apl5WStw0Q6d3F0yU4hxcwZ3QtCbW6gdAFwKdA5XfmPq1A6AACEgf7syLvxeZpRCISEvHb7EnQ3AbuPVTgB+/Tnq2P7ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zMN+134y00IOPAGbHxYb1JZP3yvBcsgKEciBTox9s/U=; b=mG46Lzf0D2mCTcxzaO26AagVVIhtLpgS/hj6VmVE+9DFu4DmtX7v3m/kDdbUnFLj+XnQ6spmvVVrlzAP872pUimm3LWl7AC1PhGTWmqGVL/sEHSjtJt+RSSQ31Ah4bMfxla3gdO6wULimnBQwV3yZ5/fTzN69H8DzWMR6/OpR/g= Received: from DM6PR11MB4593.namprd11.prod.outlook.com (2603:10b6:5:2a3::8) by DM6PR11MB2650.namprd11.prod.outlook.com (2603:10b6:5:c4::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.24; Fri, 10 Apr 2020 14:39:02 +0000 Received: from DM6PR11MB4593.namprd11.prod.outlook.com ([fe80::9d18:a682:243f:6d6d]) by DM6PR11MB4593.namprd11.prod.outlook.com ([fe80::9d18:a682:243f:6d6d%3]) with mapi id 15.20.2900.015; Fri, 10 Apr 2020 14:39:02 +0000 From: "Wiles, Keith" To: Thomas Monjalon CC: "Richardson, Bruce" , "Power, Ciara" , dev , "Laatz, Kevin" , "Pattan, Reshma" , "jerinjacobk@gmail.com" , "david.marchand@redhat.com" , "mb@smartsharesystems.com" Thread-Topic: [PATCH v2 00/16] update and simplify telemetry library. Thread-Index: AQHWDchjQaQiU90Sg0y9/LABPPMO4qhvhG8AgAD/2YCAAAU4AIAB5nyA Date: Fri, 10 Apr 2020 14:39:02 +0000 Message-ID: <93132D13-8044-43A7-A817-4B526BEEBC29@intel.com> References: <20200319171907.60891-1-ciara.power@intel.com> <2104953.NgBsaNRSFp@thomas> <20200409091909.GC605@bricha3-MOBL.ger.corp.intel.com> <8682539.rMLUfLXkoz@thomas> In-Reply-To: <8682539.rMLUfLXkoz@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=keith.wiles@intel.com; x-originating-ip: [192.55.52.207] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 159d44ed-02bf-444a-5808-08d7dd5ce9f6 x-ms-traffictypediagnostic: DM6PR11MB2650: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0369E8196C x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4593.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(136003)(366004)(376002)(396003)(346002)(39860400002)(26005)(66946007)(64756008)(66476007)(6486002)(66446008)(76116006)(66556008)(71200400001)(33656002)(5660300002)(6512007)(4326008)(8676002)(81156014)(8936002)(6916009)(2906002)(54906003)(316002)(6506007)(53546011)(966005)(86362001)(91956017)(186003)(36756003)(2616005)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: cKoc+6PgxIYy9yDQJFgxY5iQv9XjP41PTCJuEhuDQVNonjWckMef8jNnl0p8bQgEJ/LI9B6V7O33X2dpEIswUAKaTzWNkWQtKm/iK0Zlv6U9kyFqo6/rCivc34+CRE4WQZ3m43mUUUh0qon3D/gdfL9kyZwXnBYOCLUUNiVH3Qf31tTTQUDVlw5BSO/TvN7NefYL2ZVsM0J9VihJLccUkxmSAYEJMsTJMN9feGgoxcXIQcNeXRZr0Y3pquajyxL7hN9IlY3nt0WSGIlu5C2Dh2GvvLTNWRCKddCrovEcayP5PPab6rKartA9Si92AI1g/Bbv5osAa1Ng33b9OtLa6F7DIYTp/eBmRcuoA2b7bV6UbJA+yX6ITHMY+VqGRcH78iroJDNRw/AfcoTzacvnIQDZZzcgQBWAXFC8yrB+mg1/Bn+XdjBKi7NXh/A9n/lk6patAwX+DzBWS3rwt/ZEH8Ba7hAU++8evtXJLCXsF64+XU+BlXJdMlJgknaReqdv7Y4QNKcAV9O63VQ4AuiRTg== x-ms-exchange-antispam-messagedata: SJqsST1XaMLjh5yKGHUzL+QiMcOpOD0CiYE8BORoNazXBxFdovsl31i1vJVhvc/67iFpQfppkZrfFSzlIqnvPIIKk5JhTs82+ncVyaEa5/nXnlQGFHNhPU3nsuI77KYGdvFQIYavmIBLB6WURkcSJw== Content-Type: text/plain; charset="us-ascii" Content-ID: <0C8DEB4D2584704D96584D901CA8DED1@namprd11.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 159d44ed-02bf-444a-5808-08d7dd5ce9f6 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2020 14:39:02.5103 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dbRF09iY+N/chllGgAL+sJThcBk7ZD53YcX7bxzoD+cwe9kFRcn+Nkz57mywIlvs+7P33eE8oLdv3q0TgM8yww== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2650 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > On Apr 9, 2020, at 4:37 AM, Thomas Monjalon wrote: >=20 > 09/04/2020 11:19, Bruce Richardson: >> On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: >>> 08/04/2020 18:49, Ciara Power: >>>> This patchset extensively reworks the telemetry library adding new >>>> functionality and simplifying much of the existing code, while >>>> maintaining backward compatibility. >>>>=20 >>>> This work is based on the previously sent RFC for a "process info" >>>> library: https://patchwork.dpdk.org/project/dpdk/list/?series=3D7741 >>>> However, rather than creating a new library, this patchset takes >>>> that work and merges it into the existing telemetry library, as >>>> mentioned above. >>>>=20 >>>> The telemetry library as shipped in 19.11 is based upon the metrics >>>> library and outputs all statistics based on that as a source. However, >>>> this limits the telemetry output to only port-level statistics >>>> information, rather than allowing it to be used as a general scheme fo= r >>>> telemetry information across all DPDK libraries. >>>>=20 >>>> With this patchset applied, rather than the telemetry library being >>>> responsible for pulling ethdev stats and pushing them into the metrics >>>> library for retrieval later, each library e.g. ethdev, rawdev, and eve= n >>>> the metrics library itself (for backwards compatiblity) now handle the= ir >>>> own stats. Any library or app can register a callback function with >>>> telemetry, which will be called if requested by the client connected v= ia >>>> the telemetry socket. The callback function in the library/app then >>>> formats its stats, or other data, into a JSON string, and returns it t= o >>>> telemetry to be sent to the client. >>>=20 >>> I think this is a global need in DPDK, and it is usually called RPC, >>> IPC or control messaging. >>> We had a similar need for multi-process communication, thus rte_mp IPC. >>> We also need a control channel for user configuration applications. >>> We also need to control some features like logging or tracing. >>>=20 >>> In my opinion, it is time to introduce a general control channel in DPD= K. >>> The application must be in the loop of the control mechanism. >>> Making such channel standard will ease application adoption. >>>=20 >>> Please read some comments here: >>> http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ >>>=20 >> Hi Thomas, >>=20 >> I agree that having a single control mechanism or messaging mechanism in >> DPDK would be nice to have. However, I don't believe the plans for such = a >> scheme should impact this patchset right now as the idea of a common >> channel was only first mooted about a week ago, and while there has been >> some email discussion about it, there is as yet no requirements list tha= t >> I've seen, nobody actually doing coding work on it, no rfc and most >> importantly no timeline for creating and merging such into DPDK. >=20 > Yes, this is a new idea. > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > is the first step before starting a dedicated thread to design > a generic mechanism. >=20 >> At present though, DPDK has a telemetry solution that works for the use = case >> of ethdev stats and some power management info, but requires a more gene= ral >> solution to allow monitoring tools like PMDT to introspect DPDK, and als= o >> to prove statistics for other parts of DPDK such as cryptodev, eventdev, >> and other libraries, plus the application itself if the app so desires. >=20 > Doing rework on telemetry is similar to a general control mechanism. > Can we take this opportunity to work on what we believe to be a bigger > idea? It should be done anyway, so why pushing this temporary solution? > Sometimes we need a quick answer to an urgent problem. > But I don't think telemetry is currently in such situation that > a rework in 20.05 is mandatory. Updating telemetry to be more consumable and standardize on a single method= to get stats/info out of DPDK is a clean and simple solution. Starting ove= r and creating yet another solution means we are pushing this support out a= gain and many customer are asking for this support now. The current telemetry solution in this patch gives us a great starting poin= t and going back to the drawing board is a waste of time IMO and we need so= mething now. To me this is a urgent problem we need to solve now, as I want= to push PMDT and if we keep pushing out this type of support then it will = never be upstreamed. In PMDT I believe I have resolved all of the tech boards concerns and just = waiting for this patch and a patch to PCM to push the code back to DPDK aga= in. So please let's not redesign this again. > =20 >=20