From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 36C7DA034F; Wed, 13 May 2020 14:42:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A38B91D588; Wed, 13 May 2020 14:42:55 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 9F64B1D56B for ; Wed, 13 May 2020 14:42:53 +0200 (CEST) IronPort-SDR: rlBRkdo5JAzm7622YhLdsid5tVXJJFFspcsudjJ8VY8q/gYMnTaZHuLj0u/S8JtTBNTWpJEtSw lREzelp3PZRw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2020 05:42:52 -0700 IronPort-SDR: Uet2AV0RnMJicW9K9acgJJcn5Q0jng3yjXuJUSbP0ywiNk3z+Q4+NGsCPLtCUxMLluZjLwV6Go j6ebIr2Jh3Cg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,387,1583222400"; d="scan'208";a="306774959" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by FMSMGA003.fm.intel.com with ESMTP; 13 May 2020 05:42:52 -0700 Received: from fmsmsx117.amr.corp.intel.com (10.18.116.17) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 13 May 2020 05:42:52 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx117.amr.corp.intel.com (10.18.116.17) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 13 May 2020 05:42:52 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.176) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 13 May 2020 05:42:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ICf8j+EwweWXAlrZcq83mjv7X1oiEUFEFWhpuDp6BOOAZrR0DYNn66jtRSccrvgQzaksF4HPLLwKIJqv1Z4UQVcZbclkLHSMa5MFD2KZIhjz+1VhrHarmkMFVzbp5bx8wImqh6dQZH23ivt+MEdf2TisrhVkhKNzqo9Dmv/8E03r7LXZ17xSLdVu0GYyDybAQETvU37aGuPDEygJ48Fe0hFgAf3PwaR1+hfvoln2AVXPiSS0ggIo1qUiKCuWDTukZmaZy/DjsdxhCj1SD+lb4QBCJjsOBbo8zf1pEz8HKLnhvirBzH8BHA+YjSpzZPykoKcsVCHuZRoOUX4MiS3JRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cG7CZimjnapGW0MxJg0bN7XlWzP4eDnMtaunxgamA7A=; b=mH+X0MmJdH/Azanw54zMV01gTqU8DJC/Klc40r6FShGebDqqPg6ICLB2wREUIyMSvsPITGyfKgVE2NnLpoGoaLJ5fCjcds5NgfsYAMjWHtme9UIxBL19DQ9o2lFKEtGYE5LT8eApvv2ekiQN9QLW3oc9VjgagWUr2OnV3YHSyy+GN2t+o6gUUHY8U62hpmyNBz6l30flYk0T0W0Jk3E9Ap/StMBUsDgUxpXvjiN+XrkjIjaR9BlYlw8YmUf1VH6TENbuAvT2hAfuWz6rysjirnk6BHKiZDCl0HiuJSzEFetKRu4w+m2yxyN8gOz8Tj9J8zc2CiAwOJxuK+MUDi6+dw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cG7CZimjnapGW0MxJg0bN7XlWzP4eDnMtaunxgamA7A=; b=YsXfVpDUffZ9ofVbzdPTqNPI2lCZX6ZiQSw7WIYcGLAzB5LxvbD2IvVxte0/7zoEiHgCqEvvOztQqf1jxVx+9q400IdXacFHvzyMXt3pj4RfYjybtHUwSADOZe4Jy+vEt1NKqvnlsSUy/pEBZioeijHAgch8gPVXwBisfC2Ap/U= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB2759.namprd11.prod.outlook.com (2603:10b6:a02:c8::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.20; Wed, 13 May 2020 12:42:49 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4%6]) with mapi id 15.20.2979.033; Wed, 13 May 2020 12:42:49 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal , "Nicolau, Radu" CC: Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH v3] examples/ipsec-secgw: add per core packet stats Thread-Index: AQHWI6SkbuuME/sCGUWlN/0x2cLQmKicvv3ggAenbICAAZhCAA== Date: Wed, 13 May 2020 12:42:49 +0000 Message-ID: References: <1587647245-10524-1-git-send-email-anoobj@marvell.com> <1588769253-10405-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.162] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 18c1b648-73cd-4acc-856a-08d7f73b2546 x-ms-traffictypediagnostic: BYAPR11MB2759: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0402872DA1 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1CGNRwrYnuSejXqQQVO8hB7QaIaW65ALhItZtb/q45VjRyEkoNyp5yt6vZqEZ2PxFaZ4UogrPbELrfHVYtCpgJWmGAKRQaY0zeblMzMCfN0OZF29z/wL8hQZjGrFprO/94emRGi6H2ZCaTD7X73K6ECUmtSC0ZmnJMQI64EfwuNOELfgcVQSkMGjXSpgbi6cGmatB2+ZhFayT5/I/qbFje+Go+hEbLT9ujAZlYRbWBfZ/O/r2+7srdy2V/Y9/EJQIpIfFv3wc+D2UzbtDLFxIGXV9R6ldcuC9NCvOffBo4QE7MurwOZDIcrOFbfLn7bWps0n8MiRV99hgs2nlib2qcBccSMVT1FVVEgTyzX+Kmn4BQOgJ6vsBQAybnzb4HjFuqLQ4b/yi8XeYoPTJ0GiUXrSpwbPmIdjvFQEnJ67axPW/Ky5qtKnVaEF08KeCyB/LPHejyLhgUUbFmRyG0tKvV6wHHBsBgVVqJrh1+gIDD2NmShd/VXqRkvAUsWqOl0c4/APrZFjfoeXoxCW+xTrdw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(376002)(366004)(39860400002)(346002)(396003)(136003)(33430700001)(4326008)(52536014)(9686003)(55016002)(6636002)(76116006)(33656002)(66446008)(66946007)(66556008)(54906003)(110136005)(64756008)(478600001)(66476007)(316002)(7696005)(6506007)(33440700001)(186003)(26005)(86362001)(71200400001)(5660300002)(2906002)(8936002)(8676002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: SpbK1xWKSjPtN04iRuF2tAjS58KcfefnnTNpbgF2sxvPUuXN5XulV4lGMLt7GYVgCISCU3yz0Nh68O4H047Kbt2zNO0Pg8wiqWB5Fz29SGfglq0H9lHNOxJQZq2/FQ7AqTbDMQRlaC458I1xRdKUX3+6DLdSnRBnSlC5KJfJtt/ck9IWGIj/2xNw8HGnSP+vfj/OfpxNcPgXS8MgpM73WD2mXN66nM/VC2B6y+UxoQ4bEvqgZpQw+7DDnC1avopc26OvqhXJGvAuGzjB2G0uRCcjzgbJnL3LZaSoFgfl3Cs2ObypWKluSe8DQ02lFp+GCXqNqefFtKRWDvxZcQiR4QY4Y4LiGgsv0YH9HL2zR/cF1idNb1M3Tjn/XKiwPLAs+B+GwyfLcoSOaOc3YUqRkieasLS1j6IWcgOAZI7M5E5g1wc5ECeX9nSZHVyZ/tSz3XeQbbvl6iXyYkQOph2DcVD5SGesN2Gr5UfLw6GZw5dSu1SxjN10d+5vGUVr0lh8 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 18c1b648-73cd-4acc-856a-08d7f73b2546 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 May 2020 12:42:49.4485 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: abFKZqUVjbwnHRVqrZPi8GYG/HM8o4h7m1d9QJKkSsB/hgMM+zawR6OUJgKv62hOD5DncJ6Fjjk280UIiuUjevvIHg/oo549BTZUHuHm7xA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2759 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: add per core packet stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > @@ -1099,6 +1151,10 @@ ipsec_poll_mode_worker(void) > > > const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) > > > / US_PER_S * BURST_TX_DRAIN_US; > > > struct lcore_rx_queue *rxql; > > > +#if (STATS_INTERVAL > 0) > > > + const uint64_t timer_period =3D STATS_INTERVAL * rte_get_timer_hz()= ; > > > + uint64_t timer_tsc =3D 0; > > > +#endif /* STATS_INTERVAL */ > > > > > > prev_tsc =3D 0; > > > lcore_id =3D rte_lcore_id(); > > > @@ -1159,6 +1215,19 @@ ipsec_poll_mode_worker(void) > > > drain_tx_buffers(qconf); > > > drain_crypto_buffers(qconf); > > > prev_tsc =3D cur_tsc; > > > +#if (STATS_INTERVAL > 0) > > > + if (lcore_id =3D=3D rte_get_master_lcore()) { > > > + /* advance the timer */ > > > + timer_tsc +=3D diff_tsc; > > > + > > > + /* if timer has reached its timeout */ > > > + if (unlikely(timer_tsc >=3D timer_period)) { > > > + print_stats(); > > > + /* reset the timer */ > > > + timer_tsc =3D 0; > > > + } > > > + } > > > +#endif /* STATS_INTERVAL */ > > > > I still don't understand why to do it in data-path thread. > > As I said in previous comments, in DPDK there is a control thread that = can be > > used for such house-keeping tasks. > > Why not to use it (via rte_alarm or so) and keep data-path threads less= affected. >=20 > [Anoob] From Marvell's estimates, this stats collection and reporting wil= l be expensive and so cannot be enabled by default. This is required > for analyzing the traffic distribution in cases where the performance isn= 't scaling as expected. Understood. > And this patch achieves the desired feature.=20 Ok, but why not to do it in control (house-keeping) thread? That would achieve desired goal and keep data-path unaffected. > If Intel would like to improve the approach, that can be taken up as a se= parate patch. This is not a vendor specific part.=20 You making changes in common data-path code that is used by all ipsec-secgw= users. I think it is everyone benefit (and responsibility) to keep common data-pat= h code clean, tidy and fast. If we can avoid polluting it with extra code, I don't see a reason not to d= o it.=20 >=20 > > > > > } > > > > > > for (i =3D 0; i < qconf->nb_rx_queue; ++i) { @@ -1169,8 +1238,10 > > @@ > > > ipsec_poll_mode_worker(void) > > > nb_rx =3D rte_eth_rx_burst(portid, queueid, > > > pkts, MAX_PKT_BURST); > > > > > > - if (nb_rx > 0) > > > + if (nb_rx > 0) { > > > + core_stats_update_rx(nb_rx); > > > process_pkts(qconf, pkts, nb_rx, portid); > > > + } > > > > > > /* dequeue and process completed crypto-ops */ > > > if (is_unprotected_port(portid)) > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.h > > > b/examples/ipsec-secgw/ipsec-secgw.h > > > index 4b53cb5..5b3561f 100644 > > > --- a/examples/ipsec-secgw/ipsec-secgw.h > > > +++ b/examples/ipsec-secgw/ipsec-secgw.h > > > @@ -6,6 +6,8 @@ > > > > > > #include > > > > > > +#define STATS_INTERVAL 0 > > > > Shouldn't it be: > > #ifndef STATS_INTERVAL > > #define STATS_INTERVAL 0 > > #endif > > ? >=20 > [Anoob] Will update in v4. >=20 > > > > To allow user specify statis interval via EXTRA_CFLAGS=3D'-DSTATS_INTER= VAL=3D10' > > or so. > > > > > + > > > #define NB_SOCKETS 4 > > > > > > #define MAX_PKT_BURST 32 > > > @@ -69,6 +71,17 @@ struct ethaddr_info { > > > uint64_t src, dst; > > > }; > > > > > > +#if (STATS_INTERVAL > 0) > > > +struct ipsec_core_statistics { > > > + uint64_t tx; > > > + uint64_t rx; > > > + uint64_t dropped; > > > + uint64_t burst_rx; > > > +} __rte_cache_aligned; > > > + > > > +struct ipsec_core_statistics core_statistics[RTE_MAX_LCORE]; #endif > > > +/* STATS_INTERVAL */ > > > + > > > extern struct ethaddr_info ethaddr_tbl[RTE_MAX_ETHPORTS]; > > > > > > /* Port mask to identify the unprotected ports */ @@ -85,4 +98,59 @@ > > > is_unprotected_port(uint16_t port_id) > > > return unprotected_port_mask & (1 << port_id); } > > > > > > +static inline void > > > +core_stats_update_rx(int n) > > > +{ > > > +#if (STATS_INTERVAL > 0) > > > + int lcore_id =3D rte_lcore_id(); > > > + core_statistics[lcore_id].rx +=3D n; > > > + if (n =3D=3D MAX_PKT_BURST) > > > + core_statistics[lcore_id].burst_rx +=3D n; #else > > > + RTE_SET_USED(n); > > > +#endif /* STATS_INTERVAL */ > > > +} > > > + > > > +static inline void > > > +core_stats_update_tx(int n) > > > +{ > > > +#if (STATS_INTERVAL > 0) > > > + int lcore_id =3D rte_lcore_id(); > > > + core_statistics[lcore_id].tx +=3D n; > > > +#else > > > + RTE_SET_USED(n); > > > +#endif /* STATS_INTERVAL */ > > > +} > > > + > > > +static inline void > > > +core_stats_update_drop(int n) > > > +{ > > > +#if (STATS_INTERVAL > 0) > > > + int lcore_id =3D rte_lcore_id(); > > > + core_statistics[lcore_id].dropped +=3D n; #else > > > + RTE_SET_USED(n); > > > +#endif /* STATS_INTERVAL */ > > > +} > > > + > > > +/* helper routine to free bulk of packets */ static inline void > > > +free_pkts(struct rte_mbuf *mb[], uint32_t n) { > > > + uint32_t i; > > > + > > > + for (i =3D 0; i !=3D n; i++) > > > + rte_pktmbuf_free(mb[i]); > > > + > > > + core_stats_update_drop(n); > > > +} > > > + > > > +/* helper routine to free single packet */ static inline void > > > +free_pkt(struct rte_mbuf *mb) { > > > + rte_pktmbuf_free(mb); > > > + core_stats_update_drop(1); > > > > Probably just: > > free_pkts(&mb, 1); > > ? >=20 > [Anoob] Will update in v4. >=20 > > > > > +} > > > + > > > #endif /* _IPSEC_SECGW_H_ */