DPDK usage discussions
 help / color / mirror / Atom feed
* Accuracy of rte_get_tsc_hz() compared to linux
@ 2024-09-18 22:04 Isaac Boukris
  2024-09-18 23:27 ` Stephen Hemminger
                   ` (3 more replies)
  0 siblings, 4 replies; 16+ messages in thread
From: Isaac Boukris @ 2024-09-18 22:04 UTC (permalink / raw)
  To: users

Hi,

On Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz (see lscpu output at the end).

The rte_get_tsc_hz() returns 2100000 KHz but using it causes our
timestamps to lag behind real time (roughly a sec per 10 min). I
noticed the kernel uses 2095082 KHz and in fact it gives much better
results.

dmesg:
tsc: Detected 2095.082 MHz processor

tsc_freq_khz (custom kmod to exposes kernel's tsc_khz):
cat /sys/devices/system/cpu/cpu0/tsc_freq_khz
2095082

I changed the dpdk code to print more when it initializes
eal_tsc_resolution_hz and lowered the rounding of the estimations, as
follows:

git diff
diff --git a/lib/eal/common/eal_common_timer.c
b/lib/eal/common/eal_common_timer.c
index c5c4703f15..faf1efc90f 100644
--- a/lib/eal/common/eal_common_timer.c
+++ b/lib/eal/common/eal_common_timer.c
@@ -38,7 +38,7 @@ rte_get_tsc_hz(void)
 static uint64_t
 estimate_tsc_freq(void)
 {
-#define CYC_PER_10MHZ 1E7
+#define CYC_PER_10MHZ 1E3
        EAL_LOG(WARNING, "WARNING: TSC frequency estimated roughly"
                " - clock timings may be less accurate.");
        /* assume that the rte_delay_us_sleep() will sleep for 1 second */
@@ -71,6 +71,10 @@ set_tsc_freq(void)
        if (!freq)
                freq = estimate_tsc_freq();

+       EAL_LOG(DEBUG, "TSC frequency arch ~%" PRIu64 " KHz",
get_tsc_freq_arch() / 1000);
+       EAL_LOG(DEBUG, "TSC frequency linux ~%" PRIu64 " KHz",
get_tsc_freq() / 1000);
+       EAL_LOG(DEBUG, "TSC frequency estimate ~%" PRIu64 " KHz",
estimate_tsc_freq() / 1000);
+
        EAL_LOG(DEBUG, "TSC frequency is ~%" PRIu64 " KHz", freq / 1000);
        eal_tsc_resolution_hz = freq;
        mcfg->tsc_hz = freq;
diff --git a/lib/eal/linux/eal_timer.c b/lib/eal/linux/eal_timer.c
index 1cb1e92193..9254c901b8 100644
--- a/lib/eal/linux/eal_timer.c
+++ b/lib/eal/linux/eal_timer.c
@@ -192,9 +192,9 @@ get_tsc_freq(void)
 {
 #ifdef CLOCK_MONOTONIC_RAW
 #define NS_PER_SEC 1E9
-#define CYC_PER_10MHZ 1E7
+#define CYC_PER_10MHZ 1E3

-       struct timespec sleeptime = {.tv_nsec = NS_PER_SEC / 10 }; /*
1/10 second */
+       struct timespec sleeptime = {.tv_sec = 1 }; /* 1/10 second */

        struct timespec t_start, t_end;
        uint64_t tsc_hz;


I've run the helloworld application on an isolated cpu:
taskset -c 10 ./dpdk-helloworld --log-level=lib.eal:debug --no-huge

The results are:
EAL: TSC frequency arch ~2100000 KHz
EAL: TSC frequency linux ~2095082 KHz
EAL: TSC frequency estimate ~2095346 KHz

The arch one is picked, which seems rather wrong, any way to override that?
Should we lower the estimation rounding to 1MHz or even 1KHz in the linux one?

Thoughts? Thanks!

Kernel: 4.18.0-513.9.1.el8_9.x86_64

lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Intel(R) Corporation
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
BIOS Model name:     Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
Stepping:            4
CPU MHz:             2100.000
BogoMIPS:            4200.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            22528K
NUMA node0 CPU(s):   0-15,32-47
NUMA node1 CPU(s):   16-31,48-63
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts
rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp
tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1
hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq
rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl
xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total
cqm_mbm_local dtherm ida arat pln pts pku ospke md_clear flush_l1d
arch_capabilities

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2024-09-21  6:36 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-09-18 22:04 Accuracy of rte_get_tsc_hz() compared to linux Isaac Boukris
2024-09-18 23:27 ` Stephen Hemminger
2024-09-19  9:37   ` Isaac Boukris
2024-09-19 12:26   ` Isaac Boukris
2024-09-19 13:04     ` Isaac Boukris
2024-09-19 18:33       ` Isaac Boukris
2024-09-19 21:53 ` Stephen Hemminger
2024-09-19 22:02 ` Stephen Hemminger
2024-09-19 22:21 ` Stephen Hemminger
2024-09-20  3:19   ` Isaac Boukris
2024-09-20 14:39     ` Stephen Hemminger
2024-09-20 15:06       ` Isaac Boukris
2024-09-21  6:36         ` Isaac Boukris
2024-09-20  7:26   ` David Marchand
2024-09-20 14:36     ` Stephen Hemminger
2024-09-20 15:11       ` Isaac Boukris

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).