Re: [dpdk-dev] [PATCH v2] cycles: add isb before read cntvct_el0

[Linhaifeng <haifeng.lin@huawei.com>]

> -----Original Message-----
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Tuesday, March 10, 2020 5:03 PM
> To: Linhaifeng <haifeng.lin@huawei.com>
> Cc: Gavin Hu <Gavin.Hu@arm.com>; dev@dpdk.org; thomas@monjalon.net;
> chenchanghu <chenchanghu@huawei.com>; xudingke
> <xudingke@huawei.com>; Lilijun (Jerry) <jerry.lilijun@huawei.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Steve Capper
> <Steve.Capper@arm.com>; nd <nd@arm.com>
> Subject: Re: [PATCH v2] cycles: add isb before read cntvct_el0
> 
> On Tue, Mar 10, 2020 at 1:39 PM Linhaifeng <haifeng.lin@huawei.com> wrote:
> 
> Please change the subject to  more meaningful one: Something like,
> 
> eal/arm64: fix rdtsc precise version
> 
> > We should use isb rather than dsb to sync system counter to cntvct_el0.
> 
> Please tell "why" in commit message, something like this
> 
> In order to get more accurate the cntvct_el0 reading, SW must invoke  .....
> 
> Add Fixes: as well.
> 
> # Make sure it is passing ./devtools/checkpatches.sh
> and ./devtools/check-git-log.sh
> 

How to use checkpatches, it always print the help info.

dpdk # ./devtools/check-git-log.sh 0001-eal-arm64-fix-rdtsc-precise-version.patch
dpdk # export DPDK_CHECKPATCH_PATH=/Images/linhf/dpdk
dpdk # ./devtools/checkpatches.sh 0001-eal-arm64-fix-rdtsc-precise-version.patch
usage: checkpatches.sh [-q] [-v] [-nX|-r range|patch1 [patch2] ...]]

Run Linux kernel checkpatch.pl with DPDK options.
The environment variable DPDK_CHECKPATCH_PATH must be set.

The patches to check can be from stdin, files specified on the command line,
latest git commits limited with -n option, or commits in the git range
specified with -r option (default: "origin/master..").

Cannot execute DPDK_CHECKPATCH_PATH

> >
> > Reference of linux kernel:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tre
> > e/arch/arm64/include/asm/arch_timer.h?h=v5.5#n220
> >
> > Signed-off-by: Haifeng Lin <haifeng.lin@huawei.com>
> > ---
> >  .../common/include/arch/arm/rte_atomic_64.h   |  3 +++
> >  .../common/include/arch/arm/rte_cycles_64.h   | 20
> +++++++++++++++++--
> >  2 files changed, 21 insertions(+), 2 deletions(-)
> >
> > diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > index 859ae129d..2587f98a2 100644
> > --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > @@ -21,6 +21,7 @@ extern "C" {
> >
> >  #define dsb(opt) asm volatile("dsb " #opt : : : "memory")  #define
> > dmb(opt) asm volatile("dmb " #opt : : : "memory")
> > +#define isb()    (asm volatile("isb" : : : "memory"))
> >
> >  #define rte_mb() dsb(sy)
> >
> > @@ -44,6 +45,8 @@ extern "C" {
> >
> >  #define rte_cio_rmb() dmb(oshld)
> >
> > +#define rte_isb() isb()
> 
> As David said, Don't add rte_ public symbols from header files.
> 
> > +
> >  /*------------------------ 128 bit atomic operations
> > -------------------------*/
> >
> >  #if defined(__ARM_FEATURE_ATOMICS) ||
> > defined(RTE_ARM_FEATURE_ATOMICS) diff --git
> > a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > index 68e7c7338..bc4e3f8e6 100644
> > --- a/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > +++ b/lib/librte_eal/common/include/arch/arm/rte_cycles_64.h
> > @@ -18,6 +18,7 @@ extern "C" {
> >   *   The time base for this lcore.
> >   */
> >  #ifndef RTE_ARM_EAL_RDTSC_USE_PMU
> > +
> >  /**
> >   * This call is portable to any ARMv8 architecture, however, typically
> >   * cntvct_el0 runs at <= 100MHz and it may be imprecise for some tasks.
> > @@ -59,11 +60,26 @@ rte_rdtsc(void)
> >  }
> >  #endif
> >
> > +#define arch_counter_enforce_ordering(val) do
> {                                                \
> > +       uint64_t tmp, _val = (val);
> \
> > +
> \
> > +       asm
> volatile(
>                                           \
> > +       "       eor  %0, %1, %1\n"
> \
> > +       "       add  %0, sp, %0\n"
> \
> > +       "       ldr  xzr, [%0]"
> \
> > +       : "=r" (tmp) : "r" (_val));
> \
> > +} while (0)
> 
> Please use static  inline function and change the name to
> __rte_arm64_cntvct_el0_enforce_ordering() or so
> 
> > +
> >  static inline uint64_t
> >  rte_rdtsc_precise(void)
> >  {
> > -       rte_mb();
> > -       return rte_rdtsc();
> > +       uint64_t tsc;
> > +
> > +       rte_isb();
> No need for new marco, If you think, it is needed make it as
> __rte_arm64_isb() or so
> 
> > +       tsc = rte_rdtsc();
> > +       arch_counter_enforce_ordering(tsc);
> > +       return tsc;
> >  }
> >
> >  static inline uint64_t
> > --
> > 2.24.1.windows.2