From: Jan Viktorin <viktorin@rehivetech.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: Nipun Gupta <nipun.gupta@nxp.com>, "dev@dpdk.org" <dev@dpdk.org>,
"thomas.monjalon@6wind.com" <thomas.monjalon@6wind.com>,
"jianbo.liu@linaro.org" <jianbo.liu@linaro.org>,
Hemant Agrawal <hemant.agrawal@nxp.com>
Subject: Re: [dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter
Date: Fri, 19 Aug 2016 14:24:58 +0200 [thread overview]
Message-ID: <20160819142458.42dad72b@pcviktorin.fit.vutbr.cz> (raw)
In-Reply-To: <20160819114611.GA5510@localhost.localdomain>
On Fri, 19 Aug 2016 17:16:12 +0530
Jerin Jacob <jerin.jacob@caviumnetworks.com> wrote:
> On Fri, Aug 19, 2016 at 09:43:36AM +0000, Nipun Gupta wrote:
> > Hi Jerin,
> >
>
> Hi Nipun,
>
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob
> > > Sent: Thursday, August 18, 2016 17:22
> > > To: dev@dpdk.org
> > > Cc: thomas.monjalon@6wind.com; jianbo.liu@linaro.org;
> > > viktorin@rehivetech.com; Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > Subject: [dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter
> > >
> > > Existing cntvct_el0 based rte_rdtsc() provides portable
> > > means to get wall clock counter at user space. Typically
> > > it runs at <= 100MHz.
> > >
> > > The alternative method to enable rte_rdtsc() for high resolution
> > > wall clock counter is through armv8 PMU subsystem.
> > > The PMU cycle counter runs at CPU frequency, However,
> > > access to PMU cycle counter from user space is not enabled
> > > by default in the arm64 linux kernel.
> > > It is possible to enable cycle counter at user space access
> > > by configuring the PMU from the privileged mode (kernel space).
> > >
> > > by default rte_rdtsc() implementation uses portable
> > > cntvct_el0 scheme. Application can choose the PMU based
> > > implementation with CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU
> > >
> > > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > ---
> > >
> > > The PMU based scheme useful for high accuracy performance profiling.
> > > Find below the example steps to configure the PMU based cycle counter on an
> > > armv8 machine.
> > >
> > > # git clone https://github.com/jerinjacobk/armv8_pmu_cycle_counter_el0
> > > # cd armv8_pmu_cycle_counter_el0
> > > # make
> > > # sudo insmod pmu_el0_cycle_counter.ko
> > > # cd $DPDK_DIR
> > > # make config T=arm64-armv8a-linuxapp-gcc
> > > # echo "CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU=y" >> build/.config
> > > # make -j 4
> >
> > Can we make this kernel module also a part of DPDK. May be in the linuxapp so that it is also compiled with DPDK?
>
> I thought so, Later I realized it may not be a good idea to add yet
> another out of tree module in DPDK repo and DPDK tries to get rid of
> existing out of tree modules.
This has also been my way of thinking. However, if we discover that such
kernel module would be really useful, I think we can do it.
>
> >
> > >
> > > ---
> > > .../common/include/arch/arm/rte_cycles_64.h | 33
> > > ++++++++++++++++++++++
> > > 1 file changed, 33 insertions(+)
> > >
> > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > > b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > > index 14f2612..867a946 100644
> > > --- a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > > +++ b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > > @@ -45,6 +45,11 @@ extern "C" {
> > > * @return
> > > * The time base for this lcore.
> > > */
> > > +#ifndef RTE_ARM_EAL_RDTSC_USE_PMU
> > > +/**
> > > + * This call is portable to any ARMv8 architecture, however, typically
> > > + * cntvct_el0 runs at <= 100MHz and it may be imprecise for some tasks.
> > > + */
> > > static inline uint64_t
> > > rte_rdtsc(void)
> > > {
> > > @@ -53,6 +58,34 @@ rte_rdtsc(void)
> > > asm volatile("mrs %0, cntvct_el0" : "=r" (tsc));
> > > return tsc;
> > > }
> > > +#else
> > > +/**
> > > + * This is an alternative method to enable rte_rdtsc() with high resolution
> > > + * PMU cycles counter.The cycle counter runs at cpu frequency and this scheme
> > > + * uses ARMv8 PMU subsystem to get the cycle counter at userspace, However,
> > > + * access to PMU cycle counter from user space is not enabled by default in
> > > + * arm64 linux kernel.
> > > + * It is possible to enable cycle counter at user space access by configuring
> > > + * the PMU from the privileged mode (kernel space).
> > > + *
> > > + * asm volatile("msr pmintenset_el1, %0" : : "r" ((u64)(0 << 31)));
> > > + * asm volatile("msr pmcntenset_el0, %0" :: "r" BIT(31));
> > > + * asm volatile("msr pmuserenr_el0, %0" : : "r"(BIT(0) | BIT(2)));
> > > + * asm volatile("mrs %0, pmcr_el0" : "=r" (val));
> > > + * val |= (BIT(0) | BIT(2));
> > > + * isb();
> > > + * asm volatile("msr pmcr_el0, %0" : : "r" (val));
> >
> > In your git repo I see that on cleanup the cycle count register is not disabled (PMCNTENCLR_EL0). It shall be better to disable the cycle count register too at module exit.
>
> OK
+1
I've got a private kernel driver enabling and disabling (hopefully) properly
this for ARMv7. If we'd like to merge it, I'd like to have a single module
or at least single module with 2 implementations...
I can post it if it would be helpful.
Regards
Jan
>
> >
> > > + *
> > > + */
> > > +static inline uint64_t
> > > +rte_rdtsc(void)
> > > +{
> > > + uint64_t tsc;
> > > +
> > > + asm volatile("mrs %0, pmccntr_el0" : "=r"(tsc));
> > > + return tsc;
> > > +}
> > > +#endif
> > >
> > > static inline uint64_t
> > > rte_rdtsc_precise(void)
> > > --
> > > 2.5.5
> >
> > Do you also plan to support performance monitor event counters?
>
> No. This patch was inspired by armv7 PMU scheme and its part of DPDK.
> The sole reason to add this support to catch any performance regression
> through app/test application.Other than that, I think cntvct_el0 based
> existing scheme is good enough for all the use cases.
>
> >
> > Regards,
> > Nipun
> >
--
Jan Viktorin E-mail: Viktorin@RehiveTech.com
System Architect Web: www.RehiveTech.com
RehiveTech
Brno, Czech Republic
next prev parent reply other threads:[~2016-08-19 12:25 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-18 11:51 Jerin Jacob
2016-08-19 9:43 ` Nipun Gupta
2016-08-19 11:46 ` Jerin Jacob
2016-08-19 12:24 ` Jan Viktorin [this message]
2016-08-19 12:52 ` Jerin Jacob
2016-10-04 8:42 ` Thomas Monjalon
2016-08-23 10:01 ` Hemant Agrawal
2016-10-04 8:46 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160819142458.42dad72b@pcviktorin.fit.vutbr.cz \
--to=viktorin@rehivetech.com \
--cc=dev@dpdk.org \
--cc=hemant.agrawal@nxp.com \
--cc=jerin.jacob@caviumnetworks.com \
--cc=jianbo.liu@linaro.org \
--cc=nipun.gupta@nxp.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).