From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 536EFA04A2; Tue, 12 May 2020 14:14:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5C67F1BF9D; Tue, 12 May 2020 14:14:19 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 824CC1BF8F for ; Tue, 12 May 2020 14:14:17 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 04CCBYqi032038; Tue, 12 May 2020 05:14:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=H6JP7EmfQ8X92HEq9ZcpXgFjr39KnneZmlqxOmMinzs=; b=sjDEB4oQNSe41nt4BqgtqCiOZjORhrSFXxg6g4X9Cgb4WwRJf95s15nLS9ejOcSjL6ou ilxUUdiRIWAIWoWfdz+qYM6Ks26QmLPzYl2hUNAqc0vvpJd/HZY16h0vbLSvvbHySWN/ tyULJn+lOGARTbYUsu1ZYEXg0XgA7qSiKjNe6vBz9GuOx6OCMQDF0unN8iYi6qCrmbXH Nm34+l5BxIrmlHrDgXKSMwfbzJSOBidSShF7+KRanloYiz8dNdBHBZ0bNtICKusGyZ1C xMHC0Wb4Zx1QjECf4+Pm1EJSlErm9GQpZaeWiLSp6mpnrfjnhxV5BjT342Tqxms8G70k fA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 30wsvqkn7r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 12 May 2020 05:14:16 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 12 May 2020 05:14:14 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.36.57) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 12 May 2020 05:14:14 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cS3qMZmBA/sdIrcKK08BNSccKrDgGkdGjVww1Ng/bm0eNwWVTzggsEgo2Iq1PUkOZGC4CRBmaITe2xCYGsi16dlCuXXsALmluG/cZQYZzw52mgB87U4tEBcYs9IwGTmgIujZTce2AhKc/JbT2f5UR1RVzi/9XW9L/YzwNVaDqWiwye1EXQodtHF04ifgHsy9nGgFtQnGyJKXuDIhlQdMsj4XNChfLhNBZ/s8V6HH2i1PKxZ0xLuYgKY1/0ZUULRBrVsH37djNNK/Y71Thpw03gseEi92uW8U6cT0on9ZCeTogo1z7Pu9F/w/Z8mdpZWxOXO+cMAXnjMHaI94qsp81w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H6JP7EmfQ8X92HEq9ZcpXgFjr39KnneZmlqxOmMinzs=; b=FgCBzm0yZN2qyFBYniDdsLz0052wPkgXn2+ImebxxZKjK2nlMPmGdItGs8F4vC29frHox/f+03VC2EnmWsDTSOjO3svhPX4lhdus5FfvitkvtnvdDgG9EzDIKpRl18Nt61bexusbQwo1IPR+iwwYa1vqIwftFAE3WX9YCikG5wdb/8E8RVfYoA/LU3OtsAfkBIeA/YuJ1ALwpRo0dSigjjdogTgZSWbGMxlMX7JWqaglLt4lGqERUTMr5qH4MpNLgPSDcJhytx8VjWT+D3U34Kog3GuaNrRyWG84zl3Fqo2lnGkk41kjBEXUga7s4TzOIWndPlsSjGBoY9HkF7e/gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H6JP7EmfQ8X92HEq9ZcpXgFjr39KnneZmlqxOmMinzs=; b=Gt4O91E5lPH+k9D6d3RIeEioYzY8yEzlAi5VuDzu9tzRk961pg1IEh8D7ZzIr1Pm57hSq7pS4qktOeMTG0YX32rtyciQ+8WISfqD3eRL4MhjrWGrMLTqpppzhsqogedwCCqNSYjJFtI9SWgx1KISF8ia47gx9cnFYt5H/etewj4= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (2603:10b6:208:3b::26) by MN2PR18MB3070.namprd18.prod.outlook.com (2603:10b6:208:ff::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.26; Tue, 12 May 2020 12:14:13 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::15a6:7cce:c7ce:b74f]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::15a6:7cce:c7ce:b74f%7]) with mapi id 15.20.2979.033; Tue, 12 May 2020 12:14:13 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" CC: Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH v3] examples/ipsec-secgw: add per core packet stats Thread-Index: AQHWI6SQq94R+vt4mEWeQWZM7gsa+KiczUKAgAeXo9A= Date: Tue, 12 May 2020 12:14:13 +0000 Message-ID: References: <1587647245-10524-1-git-send-email-anoobj@marvell.com> <1588769253-10405-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [111.125.205.80] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e19f3291-5882-40fe-292b-08d7f66dfc06 x-ms-traffictypediagnostic: MN2PR18MB3070: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: mVfsRzfXCUpcDOP8GHHbQE0a708RM3QTrmhKlyIuWZqZ8W6zyQDb0BVSogKz3fQG6RrtUrp6iiZn75CaXVaKla6O8znwEs4TfWGfkqaAf7cIlcXjaEkDgBJ/+plFd6guM2c5IeXTcMY+ZgnQb1acYzjqjBmhT86v4YYVs5PXZn+HlkfjifppSpynlGBwKhk5arcb3EwGjOBK0I8OanV6fTEN+sb3TR+VBRmZ6nkTqYP/iGAGofj7qCjGxmQPhvcjSo3JK5eDTPW8xVnPm+pdxqLnxkuL+FJ0GkdtbwjXN0WvZ2MwhAGzSUM8CGDpfgv5BBQ/09z0OmDFCAxpcseQtY58shRL/WEvy8gP04W/hCF7Jvp1Cb6g6jIG4XEEkI20wRckHz8ozFPXabcOPNR5IIJvxybEcVDJRGBe4+cXzWQhVxAJf8h4SGqOHR3tE1S4r6/rFrWN7fT0YQdihzn/61V4VENQ4hn5Sw9T0kOpl3q2I2yxqrYRXX18g555lq0b+Y2XHP+pmdUEB3GLBDOgEQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR18MB2877.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(39860400002)(136003)(396003)(33430700001)(9686003)(2906002)(110136005)(316002)(55236004)(54906003)(52536014)(7696005)(55016002)(53546011)(6506007)(478600001)(66446008)(33440700001)(71200400001)(64756008)(76116006)(8936002)(66946007)(8676002)(33656002)(66476007)(66556008)(26005)(4326008)(86362001)(5660300002)(186003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: WFWSdLBMWuiQUfiyJ6BIR1sUq9ebbGX0qmzFR5lPWMEkV2o+dNqGLsGcus8vUG6xAPZydD/HAkP9omHMF8aTc9pIgLB55QKcJkL18o7VwOTV2Tar4hIImjdTc3G+frpMjFtDKdiDp8w7N7LyeA6BLYpiiyhVXxcZuMkXnT1XsK8SYGhNpJiwuETJX5VYMDXr3Vmv2yxzBbQOKK0jHhx2AsUx5vnuA5mOgSJqGiUV2ebLliLvcy7bagHx+8qwbKDWwaDcK5RRyY+I+FQ4uTNcc8P2LbSmP4I9B1L1Y2CrJXPz3QaNQ/LCRsa/dqSdRrgj2SZQz/wQ7H+/f+queKKhuVxuJ3Cr7b4f2ZWjZS6a4j4UurvGVe3OkQcI/73LB0Pou9h9Qz1b7uelXA56SymQ3WuiwJ46qW1LI8Kh82LMCS4G5BIe1CGPyhJCwtTlCP6BNs67ZCNMxyyfMeq5ZvuvJeo0+MOiNdOvp1bfauZs41kLh8XrIVqwXHB84yw9oRuC Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: e19f3291-5882-40fe-292b-08d7f66dfc06 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 12:14:13.5765 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XeuQ3Cy5AZRYbnCetoipHEUMDmwYBQFEuJf9B7TmFakr6FZUB8vrZYhSpeb9+khFYvx5Yis8Bndq+TZzUYr7dQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3070 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.676 definitions=2020-05-12_03:2020-05-11, 2020-05-12 signatures=0 Subject: Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: add per core packet stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: Ananyev, Konstantin > Sent: Thursday, May 7, 2020 9:42 PM > To: Anoob Joseph ; Akhil Goyal ; > Nicolau, Radu > Cc: Narayana Prasad Raju Athreya ; dev@dpdk.org > Subject: [EXT] RE: [PATCH v3] examples/ipsec-secgw: add per core packet s= tats >=20 > External Email >=20 > ---------------------------------------------------------------------- > > @@ -1099,6 +1151,10 @@ ipsec_poll_mode_worker(void) > > const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) > > / US_PER_S * BURST_TX_DRAIN_US; > > struct lcore_rx_queue *rxql; > > +#if (STATS_INTERVAL > 0) > > + const uint64_t timer_period =3D STATS_INTERVAL * rte_get_timer_hz(); > > + uint64_t timer_tsc =3D 0; > > +#endif /* STATS_INTERVAL */ > > > > prev_tsc =3D 0; > > lcore_id =3D rte_lcore_id(); > > @@ -1159,6 +1215,19 @@ ipsec_poll_mode_worker(void) > > drain_tx_buffers(qconf); > > drain_crypto_buffers(qconf); > > prev_tsc =3D cur_tsc; > > +#if (STATS_INTERVAL > 0) > > + if (lcore_id =3D=3D rte_get_master_lcore()) { > > + /* advance the timer */ > > + timer_tsc +=3D diff_tsc; > > + > > + /* if timer has reached its timeout */ > > + if (unlikely(timer_tsc >=3D timer_period)) { > > + print_stats(); > > + /* reset the timer */ > > + timer_tsc =3D 0; > > + } > > + } > > +#endif /* STATS_INTERVAL */ >=20 > I still don't understand why to do it in data-path thread. > As I said in previous comments, in DPDK there is a control thread that ca= n be > used for such house-keeping tasks. > Why not to use it (via rte_alarm or so) and keep data-path threads less a= ffected. [Anoob] From Marvell's estimates, this stats collection and reporting will = be expensive and so cannot be enabled by default. This is required for anal= yzing the traffic distribution in cases where the performance isn't scaling= as expected. And this patch achieves the desired feature. If Intel would l= ike to improve the approach, that can be taken up as a separate patch. =20 >=20 > > } > > > > for (i =3D 0; i < qconf->nb_rx_queue; ++i) { @@ -1169,8 +1238,10 > @@ > > ipsec_poll_mode_worker(void) > > nb_rx =3D rte_eth_rx_burst(portid, queueid, > > pkts, MAX_PKT_BURST); > > > > - if (nb_rx > 0) > > + if (nb_rx > 0) { > > + core_stats_update_rx(nb_rx); > > process_pkts(qconf, pkts, nb_rx, portid); > > + } > > > > /* dequeue and process completed crypto-ops */ > > if (is_unprotected_port(portid)) > > diff --git a/examples/ipsec-secgw/ipsec-secgw.h > > b/examples/ipsec-secgw/ipsec-secgw.h > > index 4b53cb5..5b3561f 100644 > > --- a/examples/ipsec-secgw/ipsec-secgw.h > > +++ b/examples/ipsec-secgw/ipsec-secgw.h > > @@ -6,6 +6,8 @@ > > > > #include > > > > +#define STATS_INTERVAL 0 >=20 > Shouldn't it be: > #ifndef STATS_INTERVAL > #define STATS_INTERVAL 0 > #endif > ? [Anoob] Will update in v4. =20 >=20 > To allow user specify statis interval via EXTRA_CFLAGS=3D'-DSTATS_INTERVA= L=3D10' > or so. >=20 > > + > > #define NB_SOCKETS 4 > > > > #define MAX_PKT_BURST 32 > > @@ -69,6 +71,17 @@ struct ethaddr_info { > > uint64_t src, dst; > > }; > > > > +#if (STATS_INTERVAL > 0) > > +struct ipsec_core_statistics { > > + uint64_t tx; > > + uint64_t rx; > > + uint64_t dropped; > > + uint64_t burst_rx; > > +} __rte_cache_aligned; > > + > > +struct ipsec_core_statistics core_statistics[RTE_MAX_LCORE]; #endif > > +/* STATS_INTERVAL */ > > + > > extern struct ethaddr_info ethaddr_tbl[RTE_MAX_ETHPORTS]; > > > > /* Port mask to identify the unprotected ports */ @@ -85,4 +98,59 @@ > > is_unprotected_port(uint16_t port_id) > > return unprotected_port_mask & (1 << port_id); } > > > > +static inline void > > +core_stats_update_rx(int n) > > +{ > > +#if (STATS_INTERVAL > 0) > > + int lcore_id =3D rte_lcore_id(); > > + core_statistics[lcore_id].rx +=3D n; > > + if (n =3D=3D MAX_PKT_BURST) > > + core_statistics[lcore_id].burst_rx +=3D n; #else > > + RTE_SET_USED(n); > > +#endif /* STATS_INTERVAL */ > > +} > > + > > +static inline void > > +core_stats_update_tx(int n) > > +{ > > +#if (STATS_INTERVAL > 0) > > + int lcore_id =3D rte_lcore_id(); > > + core_statistics[lcore_id].tx +=3D n; > > +#else > > + RTE_SET_USED(n); > > +#endif /* STATS_INTERVAL */ > > +} > > + > > +static inline void > > +core_stats_update_drop(int n) > > +{ > > +#if (STATS_INTERVAL > 0) > > + int lcore_id =3D rte_lcore_id(); > > + core_statistics[lcore_id].dropped +=3D n; #else > > + RTE_SET_USED(n); > > +#endif /* STATS_INTERVAL */ > > +} > > + > > +/* helper routine to free bulk of packets */ static inline void > > +free_pkts(struct rte_mbuf *mb[], uint32_t n) { > > + uint32_t i; > > + > > + for (i =3D 0; i !=3D n; i++) > > + rte_pktmbuf_free(mb[i]); > > + > > + core_stats_update_drop(n); > > +} > > + > > +/* helper routine to free single packet */ static inline void > > +free_pkt(struct rte_mbuf *mb) { > > + rte_pktmbuf_free(mb); > > + core_stats_update_drop(1); >=20 > Probably just: > free_pkts(&mb, 1); > ? [Anoob] Will update in v4. =20 >=20 > > +} > > + > > #endif /* _IPSEC_SECGW_H_ */