DPDK patches and discussions
 help / color / mirror / Atom feed
From: Jim Harris <james.r.harris@intel.com>
To: dev@dpdk.org, anatoly.burakov@intel.com
Subject: [dpdk-dev] [PATCH v3] timer: use rte_mp_msg to get freq from primary process
Date: Thu, 22 Aug 2019 01:42:27 -0700	[thread overview]
Message-ID: <156646334762.14099.13593080473257757748.stgit@jrharri1-skx> (raw)

Ideally, get_tsc_freq_arch() is able to provide the
TSC rate using architecture-specific means.  When that
is not possible, DPDK reverts to calculating the
TSC rate with a 100ms nanosleep or 1s sleep.  The latter
occurs more frequently in VMs which often do not have
access to the data they need from arch-specific means
(CPUID leaf 0x15 or MSR 0xCE on x86).

In secondary processes, the extra 100ms is especially
noticeable and consumes the bulk of rte_eal_init()
execution time.  So in secondary processes, if
we cannot get the TSC rate using get_tsc_freq_arch(),
try to get the TSC rate from the primary process
instead using rte_mp_msg.  This is much faster than
100ms.

Reduces rte_eal_init() execution time in a secondary
process from 165ms to 66ms on my test system.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
---
 lib/librte_eal/common/eal_common_timer.c |   62 ++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_timer.c b/lib/librte_eal/common/eal_common_timer.c
index 145543de7..ad965455d 100644
--- a/lib/librte_eal/common/eal_common_timer.c
+++ b/lib/librte_eal/common/eal_common_timer.c
@@ -15,9 +15,17 @@
 #include <rte_log.h>
 #include <rte_cycles.h>
 #include <rte_pause.h>
+#include <rte_eal.h>
+#include <rte_errno.h>
 
 #include "eal_private.h"
 
+#define EAL_TIMER_MP "eal_timer_mp_sync"
+
+struct timer_mp_param {
+	uint64_t tsc_hz;
+};
+
 /* The frequency of the RDTSC timer resolution */
 static uint64_t eal_tsc_resolution_hz;
 
@@ -74,12 +82,58 @@ estimate_tsc_freq(void)
 	return RTE_ALIGN_MUL_NEAR(rte_rdtsc() - start, CYC_PER_10MHZ);
 }
 
+static uint64_t
+get_tsc_freq_from_primary(void)
+{
+	struct rte_mp_msg mp_req = {0};
+	struct rte_mp_reply mp_reply = {0};
+	struct timer_mp_param *r;
+	struct timespec ts = {.tv_sec = 1, .tv_nsec = 0};
+	uint64_t tsc_hz;
+
+	strcpy(mp_req.name, EAL_TIMER_MP);
+	if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) ||
+	    mp_reply.nb_received != 1) {
+		tsc_hz = 0;
+	} else {
+		r = (struct timer_mp_param *)mp_reply.msgs[0].param;
+		tsc_hz = r->tsc_hz;
+	}
+
+	free(mp_reply.msgs);
+	return tsc_hz;
+}
+
+static int
+timer_mp_primary(__attribute__((unused)) const struct rte_mp_msg *msg,
+		 const void *peer)
+{
+	struct rte_mp_msg reply = {0};
+	struct timer_mp_param *r = (struct timer_mp_param *)reply.param;
+
+	r->tsc_hz = eal_tsc_resolution_hz;
+	strcpy(reply.name, EAL_TIMER_MP);
+	reply.len_param = sizeof(*r);
+
+	return rte_mp_reply(&reply, peer);
+}
+
 void
 set_tsc_freq(void)
 {
 	uint64_t freq;
+	int rc;
 
 	freq = get_tsc_freq_arch();
+	if (!freq && rte_eal_process_type() != RTE_PROC_PRIMARY) {
+		/* We couldn't get the TSC frequency through arch-specific
+		 *  means.  If this is a secondary process, try to get the
+		 *  TSC frequency from the primary process - this will
+		 *  be much faster than get_tsc_freq() or estimate_tsc_freq()
+		 *  below.
+		 */
+		freq = get_tsc_freq_from_primary();
+	}
 	if (!freq)
 		freq = get_tsc_freq();
 	if (!freq)
@@ -87,6 +141,14 @@ set_tsc_freq(void)
 
 	RTE_LOG(DEBUG, EAL, "TSC frequency is ~%" PRIu64 " KHz\n", freq / 1000);
 	eal_tsc_resolution_hz = freq;
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+		rc = rte_mp_action_register(EAL_TIMER_MP, timer_mp_primary);
+		if (rc && rte_errno != ENOTSUP) {
+			RTE_LOG(WARNING, EAL, "Could not register mp_action - "
+				"secondary processes will calculate TSC rate "
+				"independently.\n");
+		}
+	}
 }
 
 void rte_delay_us_callback_register(void (*userfunc)(unsigned int))


             reply	other threads:[~2019-08-22 15:47 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-22  8:42 Jim Harris [this message]
2019-08-26 13:39 ` [dpdk-dev] [PATCH v4] eal: use memzone to share tsc hz with secondary processes Jim Harris
2019-08-26 13:44 ` [dpdk-dev] [PATCH v5] " Jim Harris
2019-08-27  8:14   ` Bruce Richardson
2019-08-27 12:04   ` Burakov, Anatoly
2019-08-27 12:48     ` Bruce Richardson
2019-08-28  9:01       ` Burakov, Anatoly
2019-08-27 14:00     ` Bruce Richardson
2019-08-27 16:16 ` [dpdk-dev] [PATCH v6] eal: add tsc_hz to rte_mem_config Jim Harris
2019-10-07 15:28 ` [dpdk-dev] [PATCH v6 RESEND] " Jim Harris
2019-10-08  8:38   ` Bruce Richardson
2019-10-21  8:23     ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=156646334762.14099.13593080473257757748.stgit@jrharri1-skx \
    --to=james.r.harris@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).