From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3777441C4D; Thu, 9 Feb 2023 10:45:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3D4C942D12; Thu, 9 Feb 2023 10:44:14 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id AF1F842D1D for ; Thu, 9 Feb 2023 10:44:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675935852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c4u31Nhuyrq78u8mimRYk4taYS9zFQlkvqF4I8UHpSs=; b=Cz2WMS1/A2hmg8mSn4JOtEI+hba56yOJGjtMFDsjb/B1mV3mn249nyZ0Bpj091SXqI4w6R RY940sT5csncq69M4oWAbW/MVte9sctmf0sdrDJrbFqT9D+OjyX9MgQP3Df3rsKJH1zRhD xhsMC0tStGAiRXS3vN0plSN4/RYDzmw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-260-rRc5G77gOHS5MUnJV0N6KA-1; Thu, 09 Feb 2023 04:44:08 -0500 X-MC-Unique: rRc5G77gOHS5MUnJV0N6KA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95E4038173C5; Thu, 9 Feb 2023 09:44:08 +0000 (UTC) Received: from ringo.redhat.com (unknown [10.39.208.33]) by smtp.corp.redhat.com (Postfix) with ESMTP id 93592C16022; Thu, 9 Feb 2023 09:44:07 +0000 (UTC) From: Robin Jarry To: dev@dpdk.org Cc: Robin Jarry , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Kevin Laatz Subject: [PATCH v10 2/5] eal: report applications lcore usage Date: Thu, 9 Feb 2023 10:43:51 +0100 Message-Id: <20230209094354.1377424-3-rjarry@redhat.com> In-Reply-To: <20230209094354.1377424-1-rjarry@redhat.com> References: <20221123102612.1688865-1-rjarry@redhat.com> <20230209094354.1377424-1-rjarry@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Allow applications to register a callback that will be invoked in rte_lcore_dump() and when requesting lcore info in the telemetry API. The callback is expected to return the number of TSC cycles that have passed since application start and the number of these cycles that were spent doing busy work. Signed-off-by: Robin Jarry Acked-by: Morten Brørup Reviewed-by: Kevin Laatz --- Notes: v9 -> v10: no change doc/guides/rel_notes/release_23_03.rst | 7 ++++ lib/eal/common/eal_common_lcore.c | 40 +++++++++++++++++++-- lib/eal/include/rte_lcore.h | 48 ++++++++++++++++++++++++++ lib/eal/version.map | 1 + 4 files changed, 94 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst index 1fa101c420cd..17d38d5ea264 100644 --- a/doc/guides/rel_notes/release_23_03.rst +++ b/doc/guides/rel_notes/release_23_03.rst @@ -55,6 +55,13 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added support for reporting lcore usage in applications.** + + * The ``/eal/lcore/list`` and ``/eal/lcore/info`` telemetry endpoints have + been added to provide information similar to ``rte_lcore_dump()``. + * Applications can register a callback at startup via + ``rte_lcore_register_usage_cb()`` to provide lcore usage information. + * **Updated AMD axgbe driver.** * Added multi-process support. diff --git a/lib/eal/common/eal_common_lcore.c b/lib/eal/common/eal_common_lcore.c index d45a40831393..8fcdebd87692 100644 --- a/lib/eal/common/eal_common_lcore.c +++ b/lib/eal/common/eal_common_lcore.c @@ -2,6 +2,7 @@ * Copyright(c) 2010-2014 Intel Corporation */ +#include #include #include @@ -437,20 +438,45 @@ lcore_role_str(enum rte_lcore_role_t role) } } +static rte_lcore_usage_cb lcore_usage_cb; + +void +rte_lcore_register_usage_cb(rte_lcore_usage_cb cb) +{ + lcore_usage_cb = cb; +} + static int lcore_dump_cb(unsigned int lcore_id, void *arg) { struct rte_config *cfg = rte_eal_get_configuration(); char cpuset[RTE_CPU_AFFINITY_STR_LEN]; + struct rte_lcore_usage usage; + rte_lcore_usage_cb usage_cb; + char *usage_str = NULL; FILE *f = arg; int ret; + /* The callback may not set all the fields in the structure, so clear it here. */ + memset(&usage, 0, sizeof(usage)); + /* Guard against concurrent modification of lcore_usage_cb. */ + usage_cb = lcore_usage_cb; + if (usage_cb != NULL && usage_cb(lcore_id, &usage) == 0) { + if (asprintf(&usage_str, ", busy cycles %"PRIu64"/%"PRIu64, + usage.busy_cycles, usage.total_cycles) < 0) { + return -ENOMEM; + } + } ret = eal_thread_dump_affinity(&lcore_config[lcore_id].cpuset, cpuset, sizeof(cpuset)); - fprintf(f, "lcore %u, socket %u, role %s, cpuset %s%s\n", lcore_id, + fprintf(f, "lcore %u, socket %u, role %s, cpuset %s%s%s\n", lcore_id, rte_lcore_to_socket_id(lcore_id), lcore_role_str(cfg->lcore_role[lcore_id]), - cpuset, ret == 0 ? "" : "..."); + cpuset, ret == 0 ? "" : "...", + usage_str != NULL ? usage_str : ""); + + free(usage_str); + return 0; } @@ -492,7 +518,9 @@ lcore_telemetry_info_cb(unsigned int lcore_id, void *arg) { struct rte_config *cfg = rte_eal_get_configuration(); struct lcore_telemetry_info *info = arg; + struct rte_lcore_usage usage; struct rte_tel_data *cpuset; + rte_lcore_usage_cb usage_cb; unsigned int cpu; if (lcore_id != info->lcore_id) @@ -511,6 +539,14 @@ lcore_telemetry_info_cb(unsigned int lcore_id, void *arg) rte_tel_data_add_array_int(cpuset, cpu); } rte_tel_data_add_dict_container(info->d, "cpuset", cpuset, 0); + /* The callback may not set all the fields in the structure, so clear it here. */ + memset(&usage, 0, sizeof(usage)); + /* Guard against concurrent modification of lcore_usage_cb. */ + usage_cb = lcore_usage_cb; + if (usage_cb != NULL && usage_cb(lcore_id, &usage) == 0) { + rte_tel_data_add_dict_uint(info->d, "total_cycles", usage.total_cycles); + rte_tel_data_add_dict_uint(info->d, "busy_cycles", usage.busy_cycles); + } return 0; } diff --git a/lib/eal/include/rte_lcore.h b/lib/eal/include/rte_lcore.h index 9c7865052100..30f83f4d578c 100644 --- a/lib/eal/include/rte_lcore.h +++ b/lib/eal/include/rte_lcore.h @@ -328,6 +328,54 @@ typedef int (*rte_lcore_iterate_cb)(unsigned int lcore_id, void *arg); int rte_lcore_iterate(rte_lcore_iterate_cb cb, void *arg); +/** + * lcore usage statistics. + */ +struct rte_lcore_usage { + /** + * The total amount of time that the application has been running on + * this lcore, in TSC cycles. + */ + uint64_t total_cycles; + /** + * The amount of time the application was busy, handling some + * workload on this lcore, in TSC cycles. + */ + uint64_t busy_cycles; +}; + +/** + * Callback to allow applications to report lcore usage. + * + * @param [in] lcore_id + * The lcore to consider. + * @param [out] usage + * Counters representing this lcore usage. This can never be NULL. + * @return + * - 0 if fields in usage were updated successfully. The fields that the + * application does not support must not be modified. + * - a negative value if the information is not available or if any error occurred. + */ +typedef int (*rte_lcore_usage_cb)(unsigned int lcore_id, struct rte_lcore_usage *usage); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Register a callback from an application to be called in rte_lcore_dump() and + * the /eal/lcore/info telemetry endpoint handler. Applications are expected to + * report lcore usage statistics via this callback. + * + * If a callback was already registered, it can be replaced with another callback + * or unregistered with NULL. The previously registered callback may remain in + * use for an undetermined period of time. + * + * @param cb + * The callback function. + */ +__rte_experimental +void rte_lcore_register_usage_cb(rte_lcore_usage_cb cb); + /** * List all lcores. * diff --git a/lib/eal/version.map b/lib/eal/version.map index 6523102157e2..2c05e5f2fb60 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -441,6 +441,7 @@ EXPERIMENTAL { rte_thread_join; # added in 23.03 + rte_lcore_register_usage_cb; rte_thread_set_name; }; -- 2.39.1