From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E77DA00C3; Thu, 14 May 2020 06:11:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 046A41D626; Thu, 14 May 2020 06:11:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 6D1271D5FF for ; Thu, 14 May 2020 06:11:45 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 04E49q2x028957; Wed, 13 May 2020 21:11:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=j0quJgEfYqLODnfxJ2OY2bA/otz/jf8R1A/6zjhS300=; b=WAxLhwduYpKvVu2I22sANa79AfY6ISxxPTdpQbVFHhYq3XjuuicgiwPzxsZgRaGIvYSO 2wQaLkPiSzsrOQlppaRt6vQzRO6m3DSoXcsh0KEIbcfHVL0XlDdwWn5cGIV4fszlXyjJ kTJoGT9i8gU2/WNAwIpAHF7MO7MQgGNjON6oXk2QBEWljaiAmM7qiJgf93Ha6K1OLYVu gTZA6fxM/IvUn9orUpohmTjPSFIPvV6XBzTV36UH6iiwEqc4b6xbTyARh/RBv7sjcaYH dgsnqPBuFZxyFLZgmDN1ZuGzm03FNYfqXV6UjGPlO37d92jRN3rb04fEPcpaF+yPhKPh /A== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 3100xk0exd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 13 May 2020 21:11:44 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 13 May 2020 21:11:42 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.108) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 13 May 2020 21:11:42 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dMDfOn2Q6qS4gLvaxw/Mk+qTblNmpkHhtEuasIVfq2a8Sp3I/6iBcpbspwZ8oHxX6xR8Pdjv1RWvac0tExC3hae/KegJY1IV2CUv/gICuea8aVHvQtunsu+JMAIZEJcl8DpONw5pP/9Ag6rVKZvNrkniun7T0Q02VfsSNClruAlC7p5h62MqyggoaYkvQhvI6B4Trpb2apD3J+mJQe6LW06vzok1u1e4Sop4SG6sMbx/gQw438q8PbBmDXiQ7gg8yUP6/sTvExIhrxAat50iVaePw8gFHKWWeDNpDB+GUMUp1k62KNlXloTsrj1t+aJjdy74xJNlnBx0ApUkTWoZxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j0quJgEfYqLODnfxJ2OY2bA/otz/jf8R1A/6zjhS300=; b=ivbkAVGmZAh+aGINu/BFVQyLrA4ZafRqEeQGqT57dx3va85Ip1SEzJKq/vjpbPqhkay5hJQtf170Ph44WHrlRYhHQduOotN3GOY232EXmeDYL6NmZtuGvn0MBADbyzNQdRWPe90c7kbQHaF/3MZ9xMlWS/9ctZiJdbZcY+TKXhgC+zM1UQwECxQm68H2eMQMRAhby9yQe9DN5H03xvidb7UrcMDRTbY1mFmfeOugxydxsXz40C8EXi0sxG9VHZqd6+dfccSUGvnJtk7s/KGLrvtqy9bWbJwOvrzFB9RLF0zNXvIO0Cfs6D/Hg3aD3ht7L9lFnxqWiiiiOcMqYtGcqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j0quJgEfYqLODnfxJ2OY2bA/otz/jf8R1A/6zjhS300=; b=b0j09hussPVM0kt7woR8q3RjcUZ5GrM2ATT/LypoUCOxFNaA+Lr5CtZeyOxRkmRJQxUwE31kGjXALkQd0IcDdj9lQ1No5U2oQF9LMunkO/zFQahdfG+8MC9s4ZVLlhWITbo6D4KGv4eJjTDoVlGVTbAVjrDIbt9zzfSzvVZ3zDQ= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (2603:10b6:208:3b::26) by MN2PR18MB2958.namprd18.prod.outlook.com (2603:10b6:208:ff::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.34; Thu, 14 May 2020 04:11:40 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::15a6:7cce:c7ce:b74f]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::15a6:7cce:c7ce:b74f%7]) with mapi id 15.20.3000.022; Thu, 14 May 2020 04:11:40 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" CC: Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH v3] examples/ipsec-secgw: add per core packet stats Thread-Index: AQHWI6SQq94R+vt4mEWeQWZM7gsa+KiczUKAgAeXo9CAAZvWgIABAn8w Date: Thu, 14 May 2020 04:11:39 +0000 Message-ID: References: <1587647245-10524-1-git-send-email-anoobj@marvell.com> <1588769253-10405-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [111.125.205.80] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 61bace83-9e4a-4271-3361-08d7f7bce771 x-ms-traffictypediagnostic: MN2PR18MB2958: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 040359335D x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: guzfzRP5WSEIhCKwcbryq4k/HPv8lu3kyEA5rH7eRbmHy+lMAAK3LePclcJePidYixub2F25X3POVUC4HQLyHfWeKGSkeGt+dc9g14cAxjJq+yj+rJHrYHXnBGfIzxGv4GogZLrDAHN8hP3Yo6poAcew7tEOUYN7zZrYWjTNwYae+z1/AoXUWD4t0EYV6wgbjflHbSfWNNRmyTaSn6QPejQ/d1I0dgYBOCEPu/oZFBGarP1F3RYa9BsdzsexkmAa5Znb7D+pq4eP9nQeirbelrNFRfcUPBRglJ8iQTAjQsQKfBZhur8AHYABoNnhkYPwCyib81JxkMygTf3IGOboWrZd5YtnnpN8VUF0uanOVGqkdZFTHd7K1cdVLxAEbnTVHPTAX5HzSsShX3z6592ohp1x2eHdrys47biJeOLfWwS32brCY/CFtA3ErkQMw+JL x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR18MB2877.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(376002)(39860400002)(366004)(136003)(346002)(6506007)(7696005)(53546011)(55016002)(9686003)(316002)(54906003)(110136005)(86362001)(26005)(186003)(55236004)(33656002)(4326008)(8676002)(8936002)(478600001)(76116006)(66556008)(66946007)(5660300002)(64756008)(66476007)(2906002)(52536014)(71200400001)(66446008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: P9RwDn7PSO2PsmjwvfhKsIl6Gf65FQN4QYPX3+vdGyFKktaQFcIP/SC/e/fTzCNrW05P+Zhk8xHcjmyRxE1Iu/4RKBWU/cieLRMnDVK8SIaL6OnxangPIpRvFWgAeuPzJDcccwdylurpbHNE4oOGFmsW7vzNqKBty332JLZYfU9JwUX/2WiDkzp1kIXiBLs6P3Tvuv3YO3C1AHY3/o+Hz7UDxRnT6qHqzBKF2QtgMdWkvFp7mIXLPWwm5m1uqRm4ECdjJ7iWtHjllpcGOoyS7T9LQdean+s/JkAOzViTLzXDMhy4TJjS+S7lhLbB/q0F2xIeDuTab2D/ER7PF6n87H6qvqwe6PJjau8YmH2IY6RZ0WaImhtL/x36qxzhcaeDzWSOl+UyX6B3E4nsCsz+RnQNzGMFXZ7/EU6Tc6ijZFgzRzH2wOy1rnf5hsduxg6cdBlgqMcTvn2xrCG+Ad1F3hlRqYNeRf5GheENa4TZ/2VdLLkPECbsjKdgYR5gQsjE Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 61bace83-9e4a-4271-3361-08d7f7bce771 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2020 04:11:40.3213 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 0WS5JTSPHVj2SSjCSinZLaGlzEH+pC621EaCFrWutHMF6hiJm8zjNc7D5sHhAZkZUh4AgnPqaTQHxov35pjiSQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2958 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.676 definitions=2020-05-13_09:2020-05-13, 2020-05-13 signatures=0 Subject: Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: add per core packet stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: Ananyev, Konstantin > Sent: Wednesday, May 13, 2020 6:13 PM > To: Anoob Joseph ; Akhil Goyal ; > Nicolau, Radu > Cc: Narayana Prasad Raju Athreya ; dev@dpdk.org > Subject: [EXT] RE: [PATCH v3] examples/ipsec-secgw: add per core packet s= tats >=20 > External Email >=20 > ---------------------------------------------------------------------- >=20 > > > > @@ -1099,6 +1151,10 @@ ipsec_poll_mode_worker(void) > > > > const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) > > > > / US_PER_S * BURST_TX_DRAIN_US; > > > > struct lcore_rx_queue *rxql; > > > > +#if (STATS_INTERVAL > 0) > > > > + const uint64_t timer_period =3D STATS_INTERVAL * rte_get_timer_hz= (); > > > > + uint64_t timer_tsc =3D 0; > > > > +#endif /* STATS_INTERVAL */ > > > > > > > > prev_tsc =3D 0; > > > > lcore_id =3D rte_lcore_id(); > > > > @@ -1159,6 +1215,19 @@ ipsec_poll_mode_worker(void) > > > > drain_tx_buffers(qconf); > > > > drain_crypto_buffers(qconf); > > > > prev_tsc =3D cur_tsc; > > > > +#if (STATS_INTERVAL > 0) > > > > + if (lcore_id =3D=3D rte_get_master_lcore()) { > > > > + /* advance the timer */ > > > > + timer_tsc +=3D diff_tsc; > > > > + > > > > + /* if timer has reached its timeout */ > > > > + if (unlikely(timer_tsc >=3D timer_period)) { > > > > + print_stats(); > > > > + /* reset the timer */ > > > > + timer_tsc =3D 0; > > > > + } > > > > + } > > > > +#endif /* STATS_INTERVAL */ > > > > > > I still don't understand why to do it in data-path thread. > > > As I said in previous comments, in DPDK there is a control thread > > > that can be used for such house-keeping tasks. > > > Why not to use it (via rte_alarm or so) and keep data-path threads le= ss > affected. > > > > [Anoob] From Marvell's estimates, this stats collection and reporting > > will be expensive and so cannot be enabled by default. This is required= for > analyzing the traffic distribution in cases where the performance isn't s= caling as > expected. >=20 > Understood. >=20 > > And this patch achieves the desired feature. >=20 > Ok, but why not to do it in control (house-keeping) thread? > That would achieve desired goal and keep data-path unaffected. >=20 > > If Intel would like to improve the approach, that can be taken up as a = separate > patch. >=20 > This is not a vendor specific part. > You making changes in common data-path code that is used by all ipsec-sec= gw > users. > I think it is everyone benefit (and responsibility) to keep common data-p= ath code > clean, tidy and fast. > If we can avoid polluting it with extra code, I don't see a reason not to= do it. [Anoob] I cannot say I've fully understood what you have suggested. I've su= bmitted v4 based on what I understood from your comments. Please have a loo= k at it. =20 >=20 > > > > > > > > > } > > > > > > > > for (i =3D 0; i < qconf->nb_rx_queue; ++i) { @@ -1169,8 +1238,10 > > > @@ > > > > ipsec_poll_mode_worker(void) > > > > nb_rx =3D rte_eth_rx_burst(portid, queueid, > > > > pkts, MAX_PKT_BURST); > > > > > > > > - if (nb_rx > 0) > > > > + if (nb_rx > 0) { > > > > + core_stats_update_rx(nb_rx); > > > > process_pkts(qconf, pkts, nb_rx, portid); > > > > + } > > > > > > > > /* dequeue and process completed crypto-ops */ > > > > if (is_unprotected_port(portid)) diff --git > > > > a/examples/ipsec-secgw/ipsec-secgw.h > > > > b/examples/ipsec-secgw/ipsec-secgw.h > > > > index 4b53cb5..5b3561f 100644 > > > > --- a/examples/ipsec-secgw/ipsec-secgw.h > > > > +++ b/examples/ipsec-secgw/ipsec-secgw.h > > > > @@ -6,6 +6,8 @@ > > > > > > > > #include > > > > > > > > +#define STATS_INTERVAL 0 > > > > > > Shouldn't it be: > > > #ifndef STATS_INTERVAL > > > #define STATS_INTERVAL 0 > > > #endif > > > ? > > > > [Anoob] Will update in v4. > > > > > > > > To allow user specify statis interval via EXTRA_CFLAGS=3D'- > DSTATS_INTERVAL=3D10' > > > or so. > > > > > > > + > > > > #define NB_SOCKETS 4 > > > > > > > > #define MAX_PKT_BURST 32 > > > > @@ -69,6 +71,17 @@ struct ethaddr_info { > > > > uint64_t src, dst; > > > > }; > > > > > > > > +#if (STATS_INTERVAL > 0) > > > > +struct ipsec_core_statistics { > > > > + uint64_t tx; > > > > + uint64_t rx; > > > > + uint64_t dropped; > > > > + uint64_t burst_rx; > > > > +} __rte_cache_aligned; > > > > + > > > > +struct ipsec_core_statistics core_statistics[RTE_MAX_LCORE]; > > > > +#endif > > > > +/* STATS_INTERVAL */ > > > > + > > > > extern struct ethaddr_info ethaddr_tbl[RTE_MAX_ETHPORTS]; > > > > > > > > /* Port mask to identify the unprotected ports */ @@ -85,4 +98,59 > > > > @@ is_unprotected_port(uint16_t port_id) > > > > return unprotected_port_mask & (1 << port_id); } > > > > > > > > +static inline void > > > > +core_stats_update_rx(int n) > > > > +{ > > > > +#if (STATS_INTERVAL > 0) > > > > + int lcore_id =3D rte_lcore_id(); > > > > + core_statistics[lcore_id].rx +=3D n; > > > > + if (n =3D=3D MAX_PKT_BURST) > > > > + core_statistics[lcore_id].burst_rx +=3D n; #else > > > > + RTE_SET_USED(n); > > > > +#endif /* STATS_INTERVAL */ > > > > +} > > > > + > > > > +static inline void > > > > +core_stats_update_tx(int n) > > > > +{ > > > > +#if (STATS_INTERVAL > 0) > > > > + int lcore_id =3D rte_lcore_id(); > > > > + core_statistics[lcore_id].tx +=3D n; #else > > > > + RTE_SET_USED(n); > > > > +#endif /* STATS_INTERVAL */ > > > > +} > > > > + > > > > +static inline void > > > > +core_stats_update_drop(int n) > > > > +{ > > > > +#if (STATS_INTERVAL > 0) > > > > + int lcore_id =3D rte_lcore_id(); > > > > + core_statistics[lcore_id].dropped +=3D n; #else > > > > + RTE_SET_USED(n); > > > > +#endif /* STATS_INTERVAL */ > > > > +} > > > > + > > > > +/* helper routine to free bulk of packets */ static inline void > > > > +free_pkts(struct rte_mbuf *mb[], uint32_t n) { > > > > + uint32_t i; > > > > + > > > > + for (i =3D 0; i !=3D n; i++) > > > > + rte_pktmbuf_free(mb[i]); > > > > + > > > > + core_stats_update_drop(n); > > > > +} > > > > + > > > > +/* helper routine to free single packet */ static inline void > > > > +free_pkt(struct rte_mbuf *mb) { > > > > + rte_pktmbuf_free(mb); > > > > + core_stats_update_drop(1); > > > > > > Probably just: > > > free_pkts(&mb, 1); > > > ? > > > > [Anoob] Will update in v4. > > > > > > > > > +} > > > > + > > > > #endif /* _IPSEC_SECGW_H_ */