From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2248EA0C43; Thu, 23 Sep 2021 20:31:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E923541260; Thu, 23 Sep 2021 20:31:14 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id ABDDB41257 for ; Thu, 23 Sep 2021 20:31:12 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="287585378" X-IronPort-AV: E=Sophos;i="5.85,317,1624345200"; d="scan'208";a="287585378" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2021 11:31:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,317,1624345200"; d="scan'208";a="614135026" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga001.fm.intel.com with ESMTP; 23 Sep 2021 11:31:10 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Thu, 23 Sep 2021 11:31:10 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Thu, 23 Sep 2021 11:31:09 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Thu, 23 Sep 2021 11:31:09 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.102) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Thu, 23 Sep 2021 11:31:09 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ikSsJyDF7zzOLS6Hn5LRPRAiCuTqJ6RDY8P36dU1WN6zAoTjpPoaaqMjvUpsv4CT1lqGVdhcLk4ajXgxrmN/A4/lTPjpumOHsxVIdidz5YHFloLTGhVlPDOlZvOaIJFwLkZrLdE5UQhNiC2+tNP6qVVjAhETgK5OUSotvNnHObc0foFxgArTkPpzDnpKO2DLSiE8QC0pjXPrkWgCaxK61yB5vgDn2Zhbdpf1tXSg1d28EvwQgYVKqnZVSthKNRXlGSiJo17sRIYWAJlsDA23g1cShpIfPZM3DIdG2kiV/rspIoF4qWF5XsLiQ7jI0zydT1o1CJiyKqyFbPkmCQAvcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DQyIxAMKUJ0Vr82gkao3oLgVFW9/oHgKYavE3mul8WE=; b=E4lWB01N4McuCoWMOIwOSDkZ1K9gSxbupc0Gw5DUpy6Id6uHbA+69i/LZ/8hmFko1Lr7OkQyuxlpA37DrEsX4jBvF5UpKVUFUIHmHfRmbetzegblPlIRvlEZ6Ecuoz8UAKK+AfF5l7zWH/yoKtoCrN5nPZByFRA4AavTCgxU5BPzumkdcDjtfSL0Vw4KTq56HwKr3sjQ6dLAQ1N7tW69gGtC+ud6dexe1Dmo+iP7AmZZru1Vi6bAj03/rpvwxq9YGHE5+YlrtL/tOeYQYUvryvCu63l6w3Yv852XR33dVMknCfRz1ughuzu9uucCKHW39wi5aRFTx2NhnZUs1EwxBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DQyIxAMKUJ0Vr82gkao3oLgVFW9/oHgKYavE3mul8WE=; b=GMA7EoLqduqv8Qx33ficFkXgY7Bp6UIVoYe7rM2v+SphwIHI0Gnz7VSkK2hJYiIWxraA6sOp47LkcYjmozjhabmXIv3D5RJw6nvBnhvvhs/T3X1WENISXg/cko0IP+yQT08C7xwjS0CA5ebaQg/oTie7W6chAF+L7qE5AXDpelo= Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19) by DM6PR11MB4609.namprd11.prod.outlook.com (2603:10b6:5:28f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Thu, 23 Sep 2021 18:31:06 +0000 Received: from DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::740e:126e:c785:c8fd]) by DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::740e:126e:c785:c8fd%4]) with mapi id 15.20.4544.015; Thu, 23 Sep 2021 18:31:06 +0000 From: "Ananyev, Konstantin" To: "Nicolau, Radu" , "Iremonger, Bernard" , "Medvedkin, Vladimir" , Ray Kinsella CC: "dev@dpdk.org" , "Richardson, Bruce" , "Zhang, Roy Fan" , "hemant.agrawal@nxp.com" , "gakhil@marvell.com" , "anoobj@marvell.com" , "Doherty, Declan" , "Sinha, Abhijit" , "Buckley, Daniel M" , "marchana@marvell.com" , "ktejasree@marvell.com" , "matan@nvidia.com" Thread-Topic: [PATCH v6 08/10] ipsec: add support for SA telemetry Thread-Index: AQHXq6YYlbY0ow6CskWgtBRhnHEQcqux5cdQ Date: Thu, 23 Sep 2021 18:31:06 +0000 Message-ID: References: <20210713133542.3550525-1-radu.nicolau@intel.com> <20210917091747.1528262-1-radu.nicolau@intel.com> <20210917091747.1528262-9-radu.nicolau@intel.com> In-Reply-To: <20210917091747.1528262-9-radu.nicolau@intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6d90664c-5865-43ae-c6e4-08d97ec04e68 x-ms-traffictypediagnostic: DM6PR11MB4609: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: NM3hyAqTPeb4ESmo1TKwKZwdE1PP874oS3bOY0IwUmBTK8qpapAa09CC4Gs0U0BKf954BCJ6fbsPyD6dVr+iDF+itrsX9gXD45i9ofsUje7vry/iDAh+fTW+f6vgEmYDaOtTOVJ309vexh5O1qB7HJSXe1AvwaKhYtQRdKOUc5sJFPX0oTKBm+1KvFEhdp3GsC/3PDp7ncGUnmIUgS3p9fFVljcsBbH1L77VvmaNGso2u6vLHumnCy3ub88b/iv2uY7UwlV+ShcXlwdOOelJOo0xda+tXNZj2BFqgzvKRj9ElvkGE+A2ZPTon538hGnLWT89hsvsDgYcYHQDfPFPf+xWHjspzKRcvwvT+3DM1Hd43VP+g6QtraJUHmA6XLh/la/L1vG1VEsuUJ4z/WRHkoUfH/ZNH0dU243Ol5ZM7CuxttlVi1j6GSejmvpgdh/Yx2P4T2DmWDE291qEj981/80XPh4i9/LByFgAtLub7SVf2WbNMlfXKpP7h9dThgCAZVtoLsEYzHjGdI3HVlEwGiIw3aNqMJU6wQtGYFGd5CQq0v+u0NdELJwojR0wA/XloUfU6ndN8wtfGcKTBk2ZNf0aaF0Sh1BFibPgpExhH8mAob6NMYhxVhPUQVtXqc98D8Hohm+k0d0c/i/gg57tloqFAF38n0zS+BNEj8tgeaIHx/DAEMHsPrsm1LO0WwZx4AdUcijoDnSpuiab6d40JQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(66556008)(64756008)(86362001)(38070700005)(38100700002)(30864003)(33656002)(316002)(83380400001)(122000001)(66446008)(66476007)(66946007)(55016002)(76116006)(110136005)(54906003)(7696005)(186003)(71200400001)(2906002)(4326008)(55236004)(6506007)(52536014)(8936002)(5660300002)(508600001)(8676002)(26005)(9686003); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?VU/25jeBnQ01imbMmeDtf3LD82Bl/kQc+RW1vSWdf8vtCapm1jAjar2geMXV?= =?us-ascii?Q?qB7Z7o6w+rSOyhWGcGUCbo5DxAlEcfOjD725K/H7V8kBkGKgJMPerWYCE2YD?= =?us-ascii?Q?iCD8DcUC2ikB7ZFCe6xBFxa1DyTxZWQLGlQahOsGFKzC0mWxcQ5mWWcSWlWC?= =?us-ascii?Q?VzLP9iQrvnGhz7EvLa53zoCpmKFdz1P+Qffg4G1BvPWau4dsDldMmJPJasI6?= =?us-ascii?Q?/rqGI4JHqEc3S9H/DzTMLCheopkaHDv+IqhjOZ1GTwkelnyCpWrvvaGTuXsp?= =?us-ascii?Q?yBBe1jxei0k3ZuD8IVzx/a93CvRBwUnkW+qDNTluETVZ3PvQClF7dlH6dPEy?= =?us-ascii?Q?xYrEbgOmyjD31Ex+5cyCLqJQYJ868Y3weBfjAorKbDs+7GHU4vYD1UnGhS2N?= =?us-ascii?Q?zpVGvGedelJctcIVQpXufMRmKLC0nYfETNYY7hralPkKVjPGn2NyQr1ISK3I?= =?us-ascii?Q?WgIvaQw52atdEA5IIG9TkEwpE4xVbzUIB7pnnPnyg4ve4mGySYGiwr+hlZPK?= =?us-ascii?Q?E/qh6DxZHtetLQORVZ7bVJubRax58JOJxTa7c/wp5Apv8+Y2UKWqvpLtwVrm?= =?us-ascii?Q?FyyGe6zAlkvcZhZ4PAaVc8uQ4rC0Hy1a/EyurMps+kSCXYVgwRVlk95DkaVw?= =?us-ascii?Q?ecz/3x10TDl+mj+8cX46oNdaZPJ5fddis06tQoCnZIEoG/+/v9AyFJSqdG6g?= =?us-ascii?Q?u/UMwzfFyPfuj626f+HsukXQ13xuGt3tNfOSd2g/diIN/qu8/wgjUowPbC+P?= =?us-ascii?Q?gN8lKHoOtyHE7VdHZAxVolzNU0HLI2PxgoCpdXXYsNQVcdcDkbDJnmjbOpp5?= =?us-ascii?Q?hxK8hpaSEg0M+/Moeg0XSDVyShmehJOKlefGFt4tDRWnrkOImG6zwwVDITrd?= =?us-ascii?Q?6YEKqK8+dVao20JKFqkOVRyTifka9sw/Adf/4h9piTM+2gr2nkYG70wATlh1?= =?us-ascii?Q?87+tQAJ5KOvh/nZ6oofrt/zNkR6DBBmnnqYdmGAmK3d5lz1iZ4SdSXwEZ9Dg?= =?us-ascii?Q?5mvASJ+kj2XXr7MVtvYdtzyBMw1d4u4MHJLBIVh12Db3EjRgOCV++Gdc9Bq7?= =?us-ascii?Q?7cnHL4G4u1U3lvEgUuY7mW1dqMiz6/+sC2dbx0colwuvLJ0DrvxkM0lPH8ry?= =?us-ascii?Q?u+UCVZlyhfRfgiqLn66d3fvJfFKwi7sVQyJAn2q2Bjl5igxy0GxVQnmWOMHn?= =?us-ascii?Q?z3N22GjUfUf8xdph+nJr2RdSHwy32xsFXN+bn1c8z7gWSAs4u+U1P8TKkj6+?= =?us-ascii?Q?i0gDea3qYEy78/p+j2mTIiyZW79wAYI3WDmI+kywuVMQ6LdDLG5Avv1lUMCC?= =?us-ascii?Q?chgAFi5n+j1SqYzwb6eqyqkR?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6d90664c-5865-43ae-c6e4-08d97ec04e68 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Sep 2021 18:31:06.2286 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XK5ReMUS47Q0Ah+au4Qyqo6cVnInnYO/+ZiVwd0eVyBODtrFGU1D8cU+xc13IkSAwGMKp+D1IFeX58FE2xJo+X6/pX3FMt/9QNs0haM+RBg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4609 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v6 08/10] ipsec: add support for SA telemetry X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" =20 > Signed-off-by: Declan Doherty > Signed-off-by: Radu Nicolau > Signed-off-by: Abhijit Sinha > Signed-off-by: Daniel Martin Buckley > Acked-by: Fan Zhang > --- > lib/ipsec/esp_inb.c | 1 + > lib/ipsec/esp_outb.c | 12 +- > lib/ipsec/meson.build | 2 +- > lib/ipsec/rte_ipsec.h | 23 ++++ > lib/ipsec/sa.c | 255 +++++++++++++++++++++++++++++++++++++++++- > lib/ipsec/sa.h | 21 ++++ > lib/ipsec/version.map | 9 ++ > 7 files changed, 317 insertions(+), 6 deletions(-) >=20 > diff --git a/lib/ipsec/esp_inb.c b/lib/ipsec/esp_inb.c > index a6ab8fbdd5..8cb4c16302 100644 > --- a/lib/ipsec/esp_inb.c > +++ b/lib/ipsec/esp_inb.c > @@ -722,6 +722,7 @@ esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct r= te_mbuf *mb[], >=20 > /* process packets, extract seq numbers */ > k =3D process(sa, mb, sqn, dr, num, sqh_len); > + sa->statistics.count +=3D k; >=20 > /* handle unprocessed mbufs */ > if (k !=3D num && k !=3D 0) > diff --git a/lib/ipsec/esp_outb.c b/lib/ipsec/esp_outb.c > index 9fc7075796..2c02c3bb12 100644 > --- a/lib/ipsec/esp_outb.c > +++ b/lib/ipsec/esp_outb.c > @@ -617,7 +617,7 @@ uint16_t > esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf= *mb[], > uint16_t num) > { > - uint32_t i, k, icv_len, *icv; > + uint32_t i, k, icv_len, *icv, bytes; > struct rte_mbuf *ml; > struct rte_ipsec_sa *sa; > uint32_t dr[num]; > @@ -626,10 +626,12 @@ esp_outb_sqh_process(const struct rte_ipsec_session= *ss, struct rte_mbuf *mb[], >=20 > k =3D 0; > icv_len =3D sa->icv_len; > + bytes =3D 0; >=20 > for (i =3D 0; i !=3D num; i++) { > if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) =3D=3D 0) { > ml =3D rte_pktmbuf_lastseg(mb[i]); > + bytes +=3D mb[i]->data_len; Shouldn't it be pkt_len? > /* remove high-order 32 bits of esn from packet len */ > mb[i]->pkt_len -=3D sa->sqh_len; > ml->data_len -=3D sa->sqh_len; > @@ -640,6 +642,8 @@ esp_outb_sqh_process(const struct rte_ipsec_session *= ss, struct rte_mbuf *mb[], > } else > dr[i - k] =3D i; > } > + sa->statistics.count +=3D k; > + sa->statistics.bytes +=3D bytes - (sa->hdr_len * k); I don't think you need to do multiplication here. It can be postponed for reporting phase (sa->hdr_len is a constant value pe= r sa). >=20 > /* handle unprocessed mbufs */ > if (k !=3D num) { > @@ -659,16 +663,19 @@ static inline void > inline_outb_mbuf_prepare(const struct rte_ipsec_session *ss, > struct rte_mbuf *mb[], uint16_t num) > { > - uint32_t i, ol_flags; > + uint32_t i, ol_flags, bytes =3D 0; Lets keep coding style consistent: please do assignment as separate stateme= nt. >=20 > ol_flags =3D ss->security.ol_flags & RTE_SECURITY_TX_OLOAD_NEED_MDATA; > for (i =3D 0; i !=3D num; i++) { >=20 > mb[i]->ol_flags |=3D PKT_TX_SEC_OFFLOAD; > + bytes +=3D mb[i]->data_len; pkt_len? > if (ol_flags !=3D 0) > rte_security_set_pkt_metadata(ss->security.ctx, > ss->security.ses, mb[i], NULL); > } > + ss->sa->statistics.count +=3D num; > + ss->sa->statistics.bytes +=3D bytes - (ss->sa->hdr_len * num); > } >=20 > /* check if packet will exceed MSS and segmentation is required */ > @@ -752,6 +759,7 @@ inline_outb_tun_pkt_process(const struct rte_ipsec_se= ssion *ss, > sqn +=3D nb_segs[i] - 1; > } >=20 > + Empty line. > /* copy not processed mbufs beyond good ones */ > if (k !=3D num && k !=3D 0) > move_bad_mbufs(mb, dr, num, num - k); > diff --git a/lib/ipsec/meson.build b/lib/ipsec/meson.build > index 1497f573bb..f5e44cfe47 100644 > --- a/lib/ipsec/meson.build > +++ b/lib/ipsec/meson.build > @@ -6,4 +6,4 @@ sources =3D files('esp_inb.c', 'esp_outb.c', 'sa.c', 'ses= .c', 'ipsec_sad.c') > headers =3D files('rte_ipsec.h', 'rte_ipsec_sa.h', 'rte_ipsec_sad.h') > indirect_headers +=3D files('rte_ipsec_group.h') >=20 > -deps +=3D ['mbuf', 'net', 'cryptodev', 'security', 'hash'] > +deps +=3D ['mbuf', 'net', 'cryptodev', 'security', 'hash', 'telemetry'] > diff --git a/lib/ipsec/rte_ipsec.h b/lib/ipsec/rte_ipsec.h > index dd60d95915..2bb52f4b8f 100644 > --- a/lib/ipsec/rte_ipsec.h > +++ b/lib/ipsec/rte_ipsec.h > @@ -158,6 +158,29 @@ rte_ipsec_pkt_process(const struct rte_ipsec_session= *ss, struct rte_mbuf *mb[], > return ss->pkt_func.process(ss, mb, num); > } >=20 > + > +struct rte_ipsec_telemetry; > + > +/** > + * Initialize IPsec library telemetry. > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_ipsec_telemetry_init(void); > + > +/** > + * Enable per SA telemetry for a specific SA. > + * @param sa > + * Pointer to the *rte_ipsec_sa* object that will have telemetry enabl= ed. > + * @return > + * 0 on success, negative value otherwise. > + */ > +__rte_experimental > +int > +rte_ipsec_telemetry_sa_add(struct rte_ipsec_sa *sa); > + Why we don't have sa_delete() here? What user supposed to do when he destroys an sa? Another question what concurrency model is implied here? > #include >=20 > #ifdef __cplusplus > diff --git a/lib/ipsec/sa.c b/lib/ipsec/sa.c > index 8e369e4618..5b55bbc098 100644 > --- a/lib/ipsec/sa.c > +++ b/lib/ipsec/sa.c > @@ -7,7 +7,7 @@ > #include > #include > #include > - > +#include As a generic one - can we move all telemetry related functions into new .c = file (sa_telemtry or so)? No point to have it here. > #include "sa.h" > #include "ipsec_sqn.h" > #include "crypto.h" > @@ -25,6 +25,7 @@ struct crypto_xform { > struct rte_crypto_aead_xform *aead; > }; >=20 > + > /* > * helper routine, fills internal crypto_xform structure. > */ > @@ -532,6 +533,249 @@ rte_ipsec_sa_size(const struct rte_ipsec_sa_prm *pr= m) > wsz =3D prm->ipsec_xform.replay_win_sz; > return ipsec_sa_size(type, &wsz, &nb); > } > +struct rte_ipsec_telemetry { > + bool initialized; Why 'initilized' is needed at all? I think there is a static initializer for list: LIST_HEAD_INITIALIZER > + LIST_HEAD(, rte_ipsec_sa) sa_list_head; > +}; > + > +#include > + > +static struct rte_ipsec_telemetry rte_ipsec_telemetry_instance =3D { > + .initialized =3D false }; > + > +static int > +handle_telemetry_cmd_ipsec_sa_list(const char *cmd __rte_unused, > + const char *params __rte_unused, > + struct rte_tel_data *data) > +{ > + struct rte_ipsec_telemetry *telemetry =3D &rte_ipsec_telemetry_instance= ; > + struct rte_ipsec_sa *sa; > + > + rte_tel_data_start_array(data, RTE_TEL_U64_VAL); > + > + LIST_FOREACH(sa, &telemetry->sa_list_head, telemetry_next) { > + rte_tel_data_add_array_u64(data, htonl(sa->spi)); Should be ntohl() I believe. BTW, why not use rte_be_to_cpu... functions here? > + } > + > + return 0; > +} > + > +/** > + * Handle IPsec SA statistics telemetry request > + * > + * Return dict of SA's with dict of key/value counters > + * > + * { > + * "SA_SPI_XX": {"count": 0, "bytes": 0, "errors": 0}, > + * "SA_SPI_YY": {"count": 0, "bytes": 0, "errors": 0} > + * } > + * > + */ > +static int > +handle_telemetry_cmd_ipsec_sa_stats(const char *cmd __rte_unused, > + const char *params, > + struct rte_tel_data *data) > +{ > + struct rte_ipsec_telemetry *telemetry =3D &rte_ipsec_telemetry_instance= ; > + struct rte_ipsec_sa *sa; > + bool user_specified_spi =3D false; > + uint32_t sa_spi; > + > + if (params) { > + user_specified_spi =3D true; > + sa_spi =3D htonl((uint32_t)atoi(params)); strtoul() would be a better choice here. Another nit - you probably don't need user_specified_spi. As I remember SPI=3D0 is a reserved value, so I think It would be enough to= : sa_spi=3D0; if (params) {sa_spi=3D..} > + } > + > + rte_tel_data_start_dict(data); > + > + LIST_FOREACH(sa, &telemetry->sa_list_head, telemetry_next) { > + char sa_name[64]; > + > + static const char *name_pkt_cnt =3D "count"; > + static const char *name_byte_cnt =3D "bytes"; > + static const char *name_error_cnt =3D "errors"; > + struct rte_tel_data *sa_data; > + > + /* If user provided SPI only get telemetry for that SA */ > + if (user_specified_spi && (sa_spi !=3D sa->spi)) > + continue; > + > + /* allocate telemetry data struct for SA telemetry */ > + sa_data =3D rte_tel_data_alloc(); > + if (!sa_data) > + return -ENOMEM; > + > + rte_tel_data_start_dict(sa_data); > + > + /* add telemetry key/values pairs */ > + rte_tel_data_add_dict_u64(sa_data, name_pkt_cnt, > + sa->statistics.count); > + > + rte_tel_data_add_dict_u64(sa_data, name_byte_cnt, > + sa->statistics.bytes); > + > + rte_tel_data_add_dict_u64(sa_data, name_error_cnt, > + sa->statistics.errors.count); > + > + /* generate telemetry label */ > + snprintf(sa_name, sizeof(sa_name), "SA_SPI_%i", htonl(sa->spi)); Again - ntohl(). > + > + /* add SA telemetry to dictionary container */ > + rte_tel_data_add_dict_container(data, sa_name, sa_data, 0); > + } > + > + return 0; > +} > + > +static int > +handle_telemetry_cmd_ipsec_sa_configuration(const char *cmd __rte_unused= , > + const char *params, > + struct rte_tel_data *data) > +{ > + struct rte_ipsec_telemetry *telemetry =3D &rte_ipsec_telemetry_instance= ; > + struct rte_ipsec_sa *sa; > + uint32_t sa_spi; > + > + if (params) > + sa_spi =3D htonl((uint32_t)atoi(params)); > + else > + return -EINVAL; > + > + rte_tel_data_start_dict(data); > + > + LIST_FOREACH(sa, &telemetry->sa_list_head, telemetry_next) { > + uint64_t mode; > + > + if (sa_spi !=3D sa->spi) > + continue; > + > + /* add SA configuration key/values pairs */ > + rte_tel_data_add_dict_string(data, "Type", > + (sa->type & RTE_IPSEC_SATP_PROTO_MASK) =3D=3D > + RTE_IPSEC_SATP_PROTO_AH ? "AH" : "ESP"); > + > + rte_tel_data_add_dict_string(data, "Direction", > + (sa->type & RTE_IPSEC_SATP_DIR_MASK) =3D=3D > + RTE_IPSEC_SATP_DIR_IB ? "Inbound" : "Outbound"); > + > + mode =3D sa->type & RTE_IPSEC_SATP_MODE_MASK; > + > + if (mode =3D=3D RTE_IPSEC_SATP_MODE_TRANS) { > + rte_tel_data_add_dict_string(data, "Mode", "Transport"); > + } else { > + rte_tel_data_add_dict_string(data, "Mode", "Tunnel"); > + > + if ((sa->type & RTE_IPSEC_SATP_NATT_MASK) =3D=3D > + RTE_IPSEC_SATP_NATT_ENABLE) { > + if (sa->type & RTE_IPSEC_SATP_MODE_TUNLV4) { > + rte_tel_data_add_dict_string(data, > + "Tunnel-Type", > + "IPv4-UDP"); > + } else if (sa->type & > + RTE_IPSEC_SATP_MODE_TUNLV6) { > + rte_tel_data_add_dict_string(data, > + "Tunnel-Type", > + "IPv4-UDP"); > + } > + } else { > + if (sa->type & RTE_IPSEC_SATP_MODE_TUNLV4) { > + rte_tel_data_add_dict_string(data, > + "Tunnel-Type", > + "IPv4-UDP"); > + } else if (sa->type & > + RTE_IPSEC_SATP_MODE_TUNLV6) { > + rte_tel_data_add_dict_string(data, > + "Tunnel-Type", > + "IPv4-UDP"); > + } > + } > + } > + > + rte_tel_data_add_dict_string(data, > + "extended-sequence-number", > + (sa->type & RTE_IPSEC_SATP_ESN_MASK) =3D=3D > + RTE_IPSEC_SATP_ESN_ENABLE ? > + "enabled" : "disabled"); > + > + if ((sa->type & RTE_IPSEC_SATP_DIR_MASK) =3D=3D > + RTE_IPSEC_SATP_DIR_IB) > + > + if (sa->sqn.inb.rsn[sa->sqn.inb.rdidx]) > + rte_tel_data_add_dict_u64(data, > + "sequence-number", > + sa->sqn.inb.rsn[sa->sqn.inb.rdidx]->sqn); > + else > + rte_tel_data_add_dict_u64(data, > + "sequence-number", 0); > + else > + rte_tel_data_add_dict_u64(data, "sequence-number", > + sa->sqn.outb); > + > + rte_tel_data_add_dict_string(data, > + "explicit-congestion-notification", > + (sa->type & RTE_IPSEC_SATP_ECN_MASK) =3D=3D > + RTE_IPSEC_SATP_ECN_ENABLE ? > + "enabled" : "disabled"); > + > + rte_tel_data_add_dict_string(data, > + "copy-DSCP", > + (sa->type & RTE_IPSEC_SATP_DSCP_MASK) =3D=3D > + RTE_IPSEC_SATP_DSCP_ENABLE ? > + "enabled" : "disabled"); > + > + rte_tel_data_add_dict_string(data, "TSO", > + sa->tso.enabled ? "enabled" : "disabled"); > + > + if (sa->tso.enabled) > + rte_tel_data_add_dict_u64(data, "TSO-MSS", sa->tso.mss); > + > + } > + > + return 0; > +} > +int > +rte_ipsec_telemetry_init(void) > +{ > + struct rte_ipsec_telemetry *telemetry =3D &rte_ipsec_telemetry_instance= ; > + int rc =3D 0; > + > + if (telemetry->initialized) > + return rc; > + > + LIST_INIT(&telemetry->sa_list_head); > + > + rc =3D rte_telemetry_register_cmd("/ipsec/sa/list", > + handle_telemetry_cmd_ipsec_sa_list, > + "Return list of IPsec Security Associations with telemetry enabled."); > + if (rc) > + return rc; > + > + rc =3D rte_telemetry_register_cmd("/ipsec/sa/stats", > + handle_telemetry_cmd_ipsec_sa_stats, > + "Returns IPsec Security Association stastistics. Parameters: int sa_sp= i"); > + if (rc) > + return rc; > + > + rc =3D rte_telemetry_register_cmd("/ipsec/sa/details", > + handle_telemetry_cmd_ipsec_sa_configuration, > + "Returns IPsec Security Association configuration. Parameters: int sa_= spi"); > + if (rc) > + return rc; > + > + telemetry->initialized =3D true; > + > + return rc; > +} > + > +int > +rte_ipsec_telemetry_sa_add(struct rte_ipsec_sa *sa) > +{ > + struct rte_ipsec_telemetry *telemetry =3D &rte_ipsec_telemetry_instance= ; > + > + LIST_INSERT_HEAD(&telemetry->sa_list_head, sa, telemetry_next); > + > + return 0; > +} >=20 > int > rte_ipsec_sa_init(struct rte_ipsec_sa *sa, const struct rte_ipsec_sa_prm= *prm, > @@ -644,19 +888,24 @@ uint16_t > pkt_flag_process(const struct rte_ipsec_session *ss, > struct rte_mbuf *mb[], uint16_t num) > { > - uint32_t i, k; > + uint32_t i, k, bytes =3D 0; > uint32_t dr[num]; >=20 > RTE_SET_USED(ss); >=20 > k =3D 0; > for (i =3D 0; i !=3D num; i++) { > - if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) =3D=3D 0) > + if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) =3D=3D 0) { > k++; > + bytes +=3D mb[i]->data_len; > + } > else > dr[i - k] =3D i; > } >=20 > + ss->sa->statistics.count +=3D k; > + ss->sa->statistics.bytes +=3D bytes - (ss->sa->hdr_len * k); > + > /* handle unprocessed mbufs */ > if (k !=3D num) { > rte_errno =3D EBADMSG; > diff --git a/lib/ipsec/sa.h b/lib/ipsec/sa.h > index 3f38921eb3..b9b7ebec5b 100644 > --- a/lib/ipsec/sa.h > +++ b/lib/ipsec/sa.h > @@ -122,9 +122,30 @@ struct rte_ipsec_sa { > uint16_t mss; > } tso; >=20 > + LIST_ENTRY(rte_ipsec_sa) telemetry_next; > + /**< list entry for telemetry enabled SA */ I am not really fond of idea to have telemetry list stuff embedded into rte= _ipsec_sa structure. Creates all sort of concurrency problem for adding/removing SA, while readi= ng telemetry data, etc. Another issue if SA is shared my multiple-processes. Instead would be much cleaner if telemetry list will contain just a pointer= to SA. Then it would be user responsibility to add del/add sa to the telelmetry li= st in an appropriate time. Also MT working model for this new API needs to be documented properly. > + > + > + RTE_MARKER cachealign_statistics __rte_cache_min_aligned; What is the reason for all these extra alignments? > + > + /* Statistics */ > + struct { > + uint64_t count; > + uint64_t bytes; > + > + struct { > + uint64_t count; > + uint64_t authentication_failed; > + } errors; > + } statistics; > + > + RTE_MARKER cachealign_tunnel_header __rte_cache_min_aligned; > + > /* template for tunnel header */ > uint8_t hdr[IPSEC_MAX_HDR_SIZE]; >=20 > + > + RTE_MARKER cachealign_tunnel_seq_num_replay_win __rte_cache_min_aligned= ; > /* > * sqn and replay window > * In case of SA handled by multiple threads *sqn* cacheline > diff --git a/lib/ipsec/version.map b/lib/ipsec/version.map > index ba8753eac4..fed6b6aba1 100644 > --- a/lib/ipsec/version.map > +++ b/lib/ipsec/version.map > @@ -19,3 +19,12 @@ DPDK_22 { >=20 > local: *; > }; > + > +EXPERIMENTAL { > + global: > + > + # added in 21.11 > + rte_ipsec_telemetry_init; > + rte_ipsec_telemetry_sa_add; > + > +}; > -- > 2.25.1