From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A20A4A00C2; Fri, 24 Apr 2020 13:15:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 311871BC24; Fri, 24 Apr 2020 13:15:56 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 3D75D493D for ; Fri, 24 Apr 2020 13:15:53 +0200 (CEST) IronPort-SDR: 9soz3cQ/3O/BRB1r/ppP03rfzmnTMAspWH7QSlE1cuwLCBS/QmN4KBx60nHtEICeFyz+DOsCp7 3z6MSpemzAmw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Apr 2020 04:15:53 -0700 IronPort-SDR: 9CBwKOiWpOzynWja93CjPZXT/llG2Q9yN3z12QJo8Q/FcaMuHlnvi+fIbDJGmqDccQ9VGRyCnb ax9HUN7MUIqQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,311,1583222400"; d="scan'208";a="335314958" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga001.jf.intel.com with ESMTP; 24 Apr 2020 04:15:53 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 24 Apr 2020 04:15:30 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.174) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 24 Apr 2020 04:14:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oAOzffmdWdnTAR4cwZZTRI4Tj1B59457ZQ7yhL4S++RMe1mxtWbagNhROw9jTuSNahfIosjNl+PaqqGM8Aq2E5SFvi9l4fDie+gn9EVBpMJleGYcJP3Z+LU4W95s8QZoggN7Lv31cREDk7+SnbceEjrp6lJCOF+0RLX1BVar+EljJsWphCl8tsVZs8QTzn2xITKXQOtwSFNnJxNBSJOTCAOefIqxcAAZ/eAAG9WdGdnl4tFfkq3Cs4QbAgmuJRvdFsHlRNzVmOqb88mT8KG3UTNFZ7BCwopBkZNaTY/Yt28mE2qpFhQWfNbUwjgOQuR742XvuhjiqSuVHw5PmfUgzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O7pz57vUrZP5gyK37ZuZUKet7MEt/QvA7iQz5JMIBnM=; b=JR3TbDV1RjE8K4zuoreXLl4YtUMIzs4PkFaJATZFy7GKomFSzKm24Ljrv32cu+BSOR/zMdjoJ90O6AfBhdAPriMy9cuV82v41OS7ZGKSb+PcS4qBm4/kuXCsmd9GFEzTf+txcuzsZd/eNPZA08jmJTPg/eI1yQkAtEq4IOH7k2H+NDgtxuOuao02qXje6QUagd2Glx0FzasM9UeLS3/cr6IyjH3wG2By5m899QuVsIK6Yc8hzWiE2x9x8txwzvdVx5bMfPRoyAlZJw+ECT2eGE20GhcXutgj6gvAgmPmJXZuqywlWZzHkcfUQazZKjU+e6Ig4ZhJzOetC7oCbuf8pQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O7pz57vUrZP5gyK37ZuZUKet7MEt/QvA7iQz5JMIBnM=; b=ZbOrkUIjVdxnMRUuLq86EcmPfaTxUDOXBXawdkd+rYvkgTxcdhDasQHBrx4N5nfxNAKQfM/+7+TmCliMfKSGS7dNPbhrAA3bNvwol6CygFWwObGKJiO3h4Ww2gyOKg1Z3E8Yds2LSFwhwOHrkdUULIwYlu0wg9rBMwforkaEm/U= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB3816.namprd11.prod.outlook.com (2603:10b6:a03:f8::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.29; Fri, 24 Apr 2020 11:14:12 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4%6]) with mapi id 15.20.2937.020; Fri, 24 Apr 2020 11:14:12 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal , "Nicolau, Radu" CC: Narayana Prasad , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2] examples/ipsec-secgw: add per core packet stats Thread-Index: AQHWGXBRHnAN9ZG/YUWBcq2OgLQJpaiIFpRg Date: Fri, 24 Apr 2020 11:14:11 +0000 Message-ID: References: <1587444633-17996-1-git-send-email-anoobj@marvell.com> <1587647245-10524-1-git-send-email-anoobj@marvell.com> In-Reply-To: <1587647245-10524-1-git-send-email-anoobj@marvell.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1d0991cf-65e8-4414-3ad9-08d7e8409e05 x-ms-traffictypediagnostic: BYAPR11MB3816: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1002; x-forefront-prvs: 03838E948C x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(346002)(366004)(39860400002)(376002)(396003)(136003)(76116006)(110136005)(30864003)(71200400001)(5660300002)(316002)(4326008)(86362001)(7696005)(54906003)(6506007)(52536014)(2906002)(6636002)(186003)(81156014)(55016002)(66476007)(8936002)(9686003)(66946007)(66446008)(66556008)(33656002)(64756008)(26005)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6/hACngFLPmr4vTZtVkNA+s4cBdFNFHqMVbdJuFaVn9wL/wPOTAuepvTg+1bzYqTvEv/AmRfkUwuSR3VBxo+9/J7VF6ENy2MyR+fEByL5WYPHbG63yzlxArLpELv/I71P4Silt8dPE88lUJy6u2xwPx0Dc2VEK/t72MgyqyWUYuRtoEsHggn+Pn0Z15KvbxLeX8ZOikHMd1z2VCyzckmDiJmRSy1hd7iYM1jsOh2CLYK9i/8Oo0VIz3JVZhpMHvpGLQYvqfxX/WSJu+0HwAMZyIvtlkDwNbrKZX/267sdwC93K3u7cl8EZdfsU4z/tgreZl5EyFVPRJaYW/mTBNIyyMUGevMB0bUBX219F4FCxggi0ZYMcsNVHlhJcbX4vZVSMeCO4X28yC0x0JDIvBNu00vbGiYIVjQiNUYSbCRU5tBa+HYUbCa9cOl9c//wCzd x-ms-exchange-antispam-messagedata: jv1FNW+16hTejs1F+YqD8qizb5q73rq82NTlYrl5jd7f55i2sU0MrORLd72+ZokdvXg1hu88DPaJZPNBHFzoWBCX5pEYCJriwjj6+Vcdw/1waCcOo5veaiLcIF2hTs0cCH5SpgcYsYQ1yF1roFRMSQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 1d0991cf-65e8-4414-3ad9-08d7e8409e05 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2020 11:14:12.0169 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: HEd75+Nz+iiu20S4v9h2On90RJq3mJzMC+4TRyETF9gv4wVpwDkj62S3u+fWG2WSFySI9aUSXViuHfy4JLDHqJZYJeS0LZvRBoB+tEIEjYo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3816 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2] examples/ipsec-secgw: add per core packet stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" =20 > Adding per core packet handling stats to analyze traffic distribution > when multiple cores are engaged. >=20 > Since aggregating the packet stats across cores would affect > performance, keeping the feature disabled using compile time flags. >=20 > Signed-off-by: Anoob Joseph > --- >=20 > v2: > * Added lookup failure cases to drop count >=20 > examples/ipsec-secgw/ipsec-secgw.c | 118 +++++++++++++++++++++++++++++= ++++-- > examples/ipsec-secgw/ipsec-secgw.h | 2 + > examples/ipsec-secgw/ipsec.c | 13 +++- > examples/ipsec-secgw/ipsec.h | 22 +++++++ > examples/ipsec-secgw/ipsec_process.c | 5 ++ > 5 files changed, 154 insertions(+), 6 deletions(-) >=20 > diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ip= sec-secgw.c > index 6d02341..db92ddc 100644 > --- a/examples/ipsec-secgw/ipsec-secgw.c > +++ b/examples/ipsec-secgw/ipsec-secgw.c > @@ -288,6 +288,61 @@ adjust_ipv6_pktlen(struct rte_mbuf *m, const struct = rte_ipv6_hdr *iph, > } > } >=20 > +#ifdef ENABLE_STATS > +static uint64_t timer_period =3D 10; /* default period is 10 seconds */ I think it is better to add user ability to control stats period. Either runtime-option, or just compile time:=20 replace ENABLE_STATS with STATS_PERIOD (0 would mean stats disabled). > + > +/* Print out statistics on packet distribution */ > +static void > +print_stats(void) > +{ > + uint64_t total_packets_dropped, total_packets_tx, total_packets_rx; > + unsigned int coreid; > + float burst_percent; > + > + total_packets_dropped =3D 0; > + total_packets_tx =3D 0; > + total_packets_rx =3D 0; > + > + const char clr[] =3D { 27, '[', '2', 'J', '\0' }; > + const char topLeft[] =3D { 27, '[', '1', ';', '1', 'H', '\0' }; > + > + /* Clear screen and move to top left */ > + printf("%s%s", clr, topLeft); Is that really needed? > + > + printf("\nCore statistics =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"); > + > + for (coreid =3D 0; coreid < RTE_MAX_LCORE; coreid++) { > + /* skip disabled cores */ > + if (rte_lcore_is_enabled(coreid) =3D=3D 0) > + continue; > + burst_percent =3D (float)(core_statistics[coreid].burst_rx * 100)/ > + core_statistics[coreid].rx; Would float be always enough here? Might better long double? > + printf("\nStatistics for core %u ------------------------------" > + "\nPackets received: %20"PRIu64 > + "\nPackets sent: %24"PRIu64 > + "\nPackets dropped: %21"PRIu64 > + "\nBurst percent: %23.2f", > + coreid, > + core_statistics[coreid].rx, > + core_statistics[coreid].tx, > + core_statistics[coreid].dropped, > + burst_percent); > + > + total_packets_dropped +=3D core_statistics[coreid].dropped; > + total_packets_tx +=3D core_statistics[coreid].tx; > + total_packets_rx +=3D core_statistics[coreid].rx; > + } > + printf("\nAggregate statistics =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D" > + "\nTotal packets received: %14"PRIu64 > + "\nTotal packets sent: %18"PRIu64 > + "\nTotal packets dropped: %15"PRIu64, > + total_packets_rx, > + total_packets_tx, > + total_packets_dropped); > + printf("\n=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D\n"); > +} > +#endif /* ENABLE_STATS */ > + > static inline void > prepare_one_packet(struct rte_mbuf *pkt, struct ipsec_traffic *t) > { > @@ -333,6 +388,7 @@ prepare_one_packet(struct rte_mbuf *pkt, struct ipsec= _traffic *t) >=20 > /* drop packet when IPv6 header exceeds first segment length */ > if (unlikely(l3len > pkt->data_len)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkt); > return; > } > @@ -350,6 +406,7 @@ prepare_one_packet(struct rte_mbuf *pkt, struct ipsec= _traffic *t) > /* Unknown/Unsupported type, drop the packet */ > RTE_LOG(ERR, IPSEC, "Unsupported packet type 0x%x\n", > rte_be_to_cpu_16(eth->ether_type)); > + core_stats_update_drop(1); > rte_pktmbuf_free(pkt); > return; > } > @@ -471,6 +528,11 @@ send_burst(struct lcore_conf *qconf, uint16_t n, uin= t16_t port) > int32_t ret; > uint16_t queueid; >=20 > +#ifdef ENABLE_STATS > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].tx +=3D n; > +#endif /* ENABLE_STATS */ Instead of polluting genric code with ifdefs, why not to introduce 2 new functions: core_stats_update_rx(), core_stats_update_tx(= ), as you did for core_stats_drop()? > + > queueid =3D qconf->tx_queue_id[port]; > m_table =3D (struct rte_mbuf **)qconf->tx_mbufs[port].m_table; >=20 > @@ -478,6 +540,9 @@ send_burst(struct lcore_conf *qconf, uint16_t n, uint= 16_t port) >=20 > ret =3D rte_eth_tx_burst(port, queueid, m_table, n); > if (unlikely(ret < n)) { > +#ifdef ENABLE_STATS > + core_statistics[lcore_id].dropped +=3D n-ret; > +#endif /* ENABLE_STATS */ You have core_stats_update_drop() for that - use it. > do { > rte_pktmbuf_free(m_table[ret]); > } while (++ret < n); > @@ -525,6 +590,7 @@ send_fragment_packet(struct lcore_conf *qconf, struct= rte_mbuf *m, > "error code: %d\n", > __func__, m->pkt_len, rte_errno); >=20 > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > return len; > } > @@ -549,8 +615,10 @@ send_single_packet(struct rte_mbuf *m, uint16_t port= , uint8_t proto) > /* need to fragment the packet */ > } else if (frag_tbl_sz > 0) > len =3D send_fragment_packet(qconf, m, port, proto); > - else > + else { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); It looks like a lot of such places... Would it be worth to unite core_stats_update_drop() and rte_pktmbuf_free(m) Into some inline function: ipsec_secgw_packet_drop(struct rte_mbuf *m[], ui= nt32_t n)=20 and use it all over such places.=20 > + } >=20 > /* enough pkts to be sent */ > if (unlikely(len =3D=3D MAX_PKT_BURST)) { > @@ -584,18 +652,21 @@ inbound_sp_sa(struct sp_ctx *sp, struct sa_ctx *sa,= struct traffic_type *ip, > continue; > } > if (res =3D=3D DISCARD) { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > continue; > } >=20 > /* Only check SPI match for processed IPSec packets */ > if (i < lim && ((m->ol_flags & PKT_RX_SEC_OFFLOAD) =3D=3D 0)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > continue; > } >=20 > sa_idx =3D res - 1; > if (!inbound_sa_check(sa, m, sa_idx)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > continue; > } > @@ -630,8 +701,10 @@ split46_traffic(struct ipsec_traffic *trf, struct rt= e_mbuf *mb[], uint32_t num) > uint8_t *, > offsetof(struct ip6_hdr, ip6_nxt)); > n6++; > - } else > + } else { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > + } > } >=20 > trf->ip4.num =3D n4; > @@ -682,11 +755,12 @@ outbound_sp(struct sp_ctx *sp, struct traffic_type = *ip, > for (i =3D 0; i < ip->num; i++) { > m =3D ip->pkts[i]; > sa_idx =3D ip->res[i] - 1; > - if (ip->res[i] =3D=3D DISCARD) > + if (ip->res[i] =3D=3D DISCARD) { > + core_stats_update_drop(1); > rte_pktmbuf_free(m); > - else if (ip->res[i] =3D=3D BYPASS) > + } else if (ip->res[i] =3D=3D BYPASS) { Looks unnecessary. > ip->pkts[j++] =3D m; > - else { > + } else { > ipsec->res[ipsec->num] =3D sa_idx; > ipsec->pkts[ipsec->num++] =3D m; > } > @@ -705,6 +779,8 @@ process_pkts_outbound(struct ipsec_ctx *ipsec_ctx, > for (i =3D 0; i < traffic->ipsec.num; i++) > rte_pktmbuf_free(traffic->ipsec.pkts[i]); >=20 > + core_stats_update_drop(traffic->ipsec.num); > + > traffic->ipsec.num =3D 0; >=20 > outbound_sp(ipsec_ctx->sp4_ctx, &traffic->ip4, &traffic->ipsec); > @@ -745,12 +821,14 @@ process_pkts_inbound_nosp(struct ipsec_ctx *ipsec_c= tx, > /* Drop any IPv4 traffic from unprotected ports */ > for (i =3D 0; i < traffic->ip4.num; i++) > rte_pktmbuf_free(traffic->ip4.pkts[i]); > + core_stats_update_drop(traffic->ip4.num); >=20 > traffic->ip4.num =3D 0; >=20 > /* Drop any IPv6 traffic from unprotected ports */ > for (i =3D 0; i < traffic->ip6.num; i++) > rte_pktmbuf_free(traffic->ip6.pkts[i]); > + core_stats_update_drop(traffic->ip6.num); >=20 > traffic->ip6.num =3D 0; >=20 > @@ -788,6 +866,7 @@ process_pkts_outbound_nosp(struct ipsec_ctx *ipsec_ct= x, > /* Drop any IPsec traffic from protected ports */ > for (i =3D 0; i < traffic->ipsec.num; i++) > rte_pktmbuf_free(traffic->ipsec.pkts[i]); > + core_stats_update_drop(traffic->ipsec.num); >=20 > n =3D 0; >=20 > @@ -901,6 +980,7 @@ route4_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *p= kts[], uint8_t nb_pkts) > } >=20 > if ((pkt_hop & RTE_LPM_LOOKUP_SUCCESS) =3D=3D 0) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -953,6 +1033,7 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *= pkts[], uint8_t nb_pkts) > } >=20 > if (pkt_hop =3D=3D -1) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -1099,6 +1180,9 @@ ipsec_poll_mode_worker(void) > const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) > / US_PER_S * BURST_TX_DRAIN_US; > struct lcore_rx_queue *rxql; > +#ifdef ENABLE_STATS > + uint64_t timer_tsc =3D 0; > +#endif /* ENABLE_STATS */ Probably better just RTE_SET_USED(timer_tsc); >=20 > prev_tsc =3D 0; > lcore_id =3D rte_lcore_id(); > @@ -1159,6 +1243,19 @@ ipsec_poll_mode_worker(void) > drain_tx_buffers(qconf); > drain_crypto_buffers(qconf); > prev_tsc =3D cur_tsc; > +#ifdef ENABLE_STATS > + if (lcore_id =3D=3D rte_get_master_lcore()) { > + /* advance the timer */ > + timer_tsc +=3D diff_tsc; > + > + /* if timer has reached its timeout */ > + if (unlikely(timer_tsc >=3D timer_period)) { > + print_stats(); > + /* reset the timer */ > + timer_tsc =3D 0; > + } > + } > +#endif /* ENABLE_STATS */ Why to do stats collection/display inside data-path? Why not use rte_timer/rte_alarm and make it happen in control thread? Another option - make stats printing at some signal to the process. In that case we don't need to bother with time period at all - it will be user to decide. Again if we remove that print_stats() from data-path it might become=20 cheap enough to always collect it, and we will not need ENABLE_STATS macro at all. > } >=20 > for (i =3D 0; i < qconf->nb_rx_queue; ++i) { > @@ -1169,6 +1266,12 @@ ipsec_poll_mode_worker(void) > nb_rx =3D rte_eth_rx_burst(portid, queueid, > pkts, MAX_PKT_BURST); >=20 > +#ifdef ENABLE_STATS > + core_statistics[lcore_id].rx +=3D nb_rx; > + if (nb_rx =3D=3D MAX_PKT_BURST) > + core_statistics[lcore_id].burst_rx +=3D nb_rx; > +#endif /* ENABLE_STATS */ > + Same for above for TX: no need to pollute the code with ifdefs. Better to introduce new function: stats_update_rx() or so. > if (nb_rx > 0) > process_pkts(qconf, pkts, nb_rx, portid); >=20 > @@ -2747,6 +2850,11 @@ main(int32_t argc, char **argv) > signal(SIGINT, signal_handler); > signal(SIGTERM, signal_handler); >=20 > +#ifdef ENABLE_STATS > + /* convert to number of cycles */ > + timer_period *=3D rte_get_timer_hz(); > +#endif /* ENABLE_STATS */ > + > /* initialize event helper configuration */ > eh_conf =3D eh_conf_init(); > if (eh_conf =3D=3D NULL) > diff --git a/examples/ipsec-secgw/ipsec-secgw.h b/examples/ipsec-secgw/ip= sec-secgw.h > index 4b53cb5..d886a35 100644 > --- a/examples/ipsec-secgw/ipsec-secgw.h > +++ b/examples/ipsec-secgw/ipsec-secgw.h > @@ -6,6 +6,8 @@ >=20 > #include >=20 > +//#define ENABLE_STATS > + Should be removed I think. > #define NB_SOCKETS 4 >=20 > #define MAX_PKT_BURST 32 > diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c > index bf88d80..dcb9312 100644 > --- a/examples/ipsec-secgw/ipsec.c > +++ b/examples/ipsec-secgw/ipsec.c > @@ -499,8 +499,10 @@ enqueue_cop_burst(struct cdev_qp *cqp) > " enqueued %u crypto ops out of %u\n", > cqp->id, cqp->qp, ret, len); > /* drop packets that we fail to enqueue */ > - for (i =3D ret; i < len; i++) > + for (i =3D ret; i < len; i++) { > + core_stats_update_drop(1); > rte_pktmbuf_free(cqp->buf[i]->sym->m_src); > + } > } > cqp->in_flight +=3D ret; > cqp->len =3D 0; > @@ -528,6 +530,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, >=20 > for (i =3D 0; i < nb_pkts; i++) { > if (unlikely(sas[i] =3D=3D NULL)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -549,6 +552,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, >=20 > if ((unlikely(ips->security.ses =3D=3D NULL)) && > create_lookaside_session(ipsec_ctx, sa, ips)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -563,6 +567,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, > case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO: > RTE_LOG(ERR, IPSEC, "CPU crypto is not supported by the" > " legacy mode."); > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; >=20 > @@ -575,6 +580,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, >=20 > if ((unlikely(ips->crypto.ses =3D=3D NULL)) && > create_lookaside_session(ipsec_ctx, sa, ips)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -584,6 +590,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, >=20 > ret =3D xform_func(pkts[i], sa, &priv->cop); > if (unlikely(ret)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -608,6 +615,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec= _ctx *ipsec_ctx, >=20 > ret =3D xform_func(pkts[i], sa, &priv->cop); > if (unlikely(ret)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkts[i]); > continue; > } > @@ -643,6 +651,7 @@ ipsec_inline_dequeue(ipsec_xform_fn xform_func, struc= t ipsec_ctx *ipsec_ctx, > sa =3D priv->sa; > ret =3D xform_func(pkt, sa, &priv->cop); > if (unlikely(ret)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkt); > continue; > } > @@ -690,12 +699,14 @@ ipsec_dequeue(ipsec_xform_fn xform_func, struct ips= ec_ctx *ipsec_ctx, > RTE_SECURITY_ACTION_TYPE_NONE) { > ret =3D xform_func(pkt, sa, cops[j]); > if (unlikely(ret)) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkt); > continue; > } > } else if (ipsec_get_action_type(sa) =3D=3D > RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) { > if (cops[j]->status) { > + core_stats_update_drop(1); > rte_pktmbuf_free(pkt); > continue; > } > diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h > index 1e642d1..8519eab 100644 > --- a/examples/ipsec-secgw/ipsec.h > +++ b/examples/ipsec-secgw/ipsec.h > @@ -46,6 +46,17 @@ >=20 > #define IP6_VERSION (6) >=20 > +#ifdef ENABLE_STATS > +struct ipsec_core_statistics { > + uint64_t tx; > + uint64_t rx; > + uint64_t dropped; > + uint64_t burst_rx; A bit strange to have burst_rx and no similar counterpart for tx. BTW, do you need burst_rx? Might be better: nb_calls_rx, nb_calls_tx; and then: rx/nb_calls_rx will give you average burst size. > +} __rte_cache_aligned; > + > +struct ipsec_core_statistics core_statistics[RTE_MAX_ETHPORTS]; Should be RTE_MAX_LCORES, I think. > +#endif /* ENABLE_STATS */ > + > struct rte_crypto_xform; > struct ipsec_xform; > struct rte_mbuf; > @@ -416,4 +427,15 @@ check_flow_params(uint16_t fdir_portid, uint8_t fdir= _qid); > int > create_ipsec_esp_flow(struct ipsec_sa *sa); >=20 > +static inline void > +core_stats_update_drop(int n) > +{ > +#ifdef ENABLE_STATS > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].dropped +=3D n; > +#else > + RTE_SET_USED(n); > +#endif /* ENABLE_STATS */ > +} > + > #endif /* __IPSEC_H__ */ > diff --git a/examples/ipsec-secgw/ipsec_process.c b/examples/ipsec-secgw/= ipsec_process.c > index bb2f2b8..05cb3ad 100644 > --- a/examples/ipsec-secgw/ipsec_process.c > +++ b/examples/ipsec-secgw/ipsec_process.c > @@ -24,6 +24,11 @@ free_pkts(struct rte_mbuf *mb[], uint32_t n) > { > uint32_t i; >=20 > +#ifdef ENABLE_STATS > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].dropped +=3D n; > +#endif /* ENABLE_STATS */ > + Same as above - why not use stats_update_drop() here? > for (i =3D 0; i !=3D n; i++) > rte_pktmbuf_free(mb[i]); > } > -- > 2.7.4