From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21BFEA0598; Fri, 10 Apr 2020 18:44:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A68D91D614; Fri, 10 Apr 2020 18:43:59 +0200 (CEST) Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by dpdk.org (Postfix) with ESMTP id 45B9D1D5F1 for ; Fri, 10 Apr 2020 18:43:56 +0200 (CEST) Received: by mail-lf1-f68.google.com with SMTP id 131so1731476lfh.11 for ; Fri, 10 Apr 2020 09:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mal3E7O5kzn2u7Zr4faoLT2/hNtxrHtmR1Zz4Rjh7O0=; b=Uv8/6wYqJ1zcWQn2PjFnQlPlSUMiwx1ypt5/5X9SOxyHcjIUywImqbqghq0p2SgR/e tI20wwAlD4nfdEuB8tvLLgOU0xD50IPZlQW88eg3LVrkedbMpDO5TzT92RRIOi05DTvi UOnpBSQdo0a1OUbPb5HOXbXCZRgvAG3coLQ7Bf1u6VEP+Iy5LVXCapr9t0r3fYN1xxpO EA0A76KKiy8WY9ESndeN+QGsNx60fAB9cLTLkmxS5YCxPdzQExQHWW6JnpWfaiCVPnlV waVAKoyv4hcNyISF9u+HjYOiVJz5krjJJ6e6IwcIYTxb7sWnQzKnNdl/XqeTSq6wpuZQ Bd/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mal3E7O5kzn2u7Zr4faoLT2/hNtxrHtmR1Zz4Rjh7O0=; b=DYN9FE4rBbGM6bmjllxHkDXq8+7tyj9VgLj3Qt1pr5Qv6pEosktSvJijoZEHPzVCqJ xn4O5VNRKilAe2iekOZogPkIKjNCwXQDYlQL0uAIUVDYbJSBZbR75bZUugNdozCHj1Bw iLdbsBp2OG9JsVayv+kExoYkQynWEhDGnfGVzHFV3BY//5jbnvaPnIMdA6Wo36Jwnq/X O+NdRqujqKbrGwJuPvzYq+I5gzzsqihBer7l/YalTQjju3uRaU1aNUy0w5UyYhcoUsfm 7aoZjuJfmU1DP3Rt7BzJc8v5sED11eHZdxE6QaBW+fgJXI/7DSOXgybLIAa0sYMpxtaj k4vw== X-Gm-Message-State: AGi0Pubc6fBJXGeJU+UPfCXKCu0bsdEJ/bTJ27oqvWU7Lwz0ZFgeqTZI UgeSMXvugQSFsI1c6oWBfoGxgvJYSyxdQA== X-Google-Smtp-Source: APiQypI50IENePABYeS22iO9KTKpKSCqETqOXgy/e9LfqaXD2zBbVY5m7wnH96vEyp8lNdy51njHaQ== X-Received: by 2002:ac2:48b1:: with SMTP id u17mr980098lfg.187.1586537035450; Fri, 10 Apr 2020 09:43:55 -0700 (PDT) Received: from localhost.localdomain (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.googlemail.com with ESMTPSA id d21sm1343075ljc.49.2020.04.10.09.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Apr 2020 09:43:54 -0700 (PDT) From: Dmitry Kozlyuk To: dev@dpdk.org Cc: "Dmitry Malloy (MESHCHANINOV)" , Narcisa Ana Maria Vasile , Fady Bader , Tal Shnaiderman , Dmitry Kozlyuk , Harini Ramakrishnan , Omar Cardona , Pallavi Kadam , Ranjit Menon , Jeff Shaw , Anand Rawat Date: Fri, 10 Apr 2020 19:43:35 +0300 Message-Id: <20200410164342.1194634-4-dmitry.kozliuk@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> References: <20200330041026.784624-1-dmitry.kozliuk@gmail.com> <20200410164342.1194634-1-dmitry.kozliuk@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 03/10] eal/windows: improve CPU and NUMA node detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 1. Map CPU cores to their respective NUMA nodes as reported by system. 2. Support systems with more than 64 cores (multiple processor groups). 3. Fix magic constants, styling issues, and compiler warnings. 4. Add EAL private function to map DPDK socket ID to NUMA node number. Fixes: 53ffd9f080fc ("eal/windows: add minimum viable code") Signed-off-by: Dmitry Kozlyuk --- lib/librte_eal/windows/eal_lcore.c | 185 +++++++++++++++++---------- lib/librte_eal/windows/eal_windows.h | 10 ++ 2 files changed, 124 insertions(+), 71 deletions(-) diff --git a/lib/librte_eal/windows/eal_lcore.c b/lib/librte_eal/windows/eal_lcore.c index 82ee45413..9d931d50a 100644 --- a/lib/librte_eal/windows/eal_lcore.c +++ b/lib/librte_eal/windows/eal_lcore.c @@ -3,103 +3,146 @@ */ #include +#include #include #include +#include +#include +#include #include "eal_private.h" #include "eal_thread.h" #include "eal_windows.h" -/* global data structure that contains the CPU map */ -static struct _wcpu_map { - unsigned int total_procs; - unsigned int proc_sockets; - unsigned int proc_cores; - unsigned int reserved; - struct _win_lcore_map { - uint8_t socket_id; - uint8_t core_id; - } wlcore_map[RTE_MAX_LCORE]; -} wcpu_map = { 0 }; - -/* - * Create a map of all processors and associated cores on the system - */ +/** Number of logical processors (cores) in a processor group (32 or 64). */ +#define EAL_PROCESSOR_GROUP_SIZE (sizeof(KAFFINITY) * CHAR_BIT) + +struct lcore_map { + uint8_t socket_id; + uint8_t core_id; +}; + +struct socket_map { + uint16_t node_id; +}; + +struct cpu_map { + unsigned int socket_count; + unsigned int lcore_count; + struct lcore_map lcores[RTE_MAX_LCORE]; + struct socket_map sockets[RTE_MAX_NUMA_NODES]; +}; + +static struct cpu_map cpu_map = { 0 }; + void -eal_create_cpu_map() +eal_create_cpu_map(void) { - wcpu_map.total_procs = - GetActiveProcessorCount(ALL_PROCESSOR_GROUPS); - - LOGICAL_PROCESSOR_RELATIONSHIP lprocRel; - DWORD lprocInfoSize = 0; - BOOL ht_enabled = FALSE; - - /* First get the processor package information */ - lprocRel = RelationProcessorPackage; - /* Determine the size of buffer we need (pass NULL) */ - GetLogicalProcessorInformationEx(lprocRel, NULL, &lprocInfoSize); - wcpu_map.proc_sockets = lprocInfoSize / 48; - - lprocInfoSize = 0; - /* Next get the processor core information */ - lprocRel = RelationProcessorCore; - GetLogicalProcessorInformationEx(lprocRel, NULL, &lprocInfoSize); - wcpu_map.proc_cores = lprocInfoSize / 48; - - if (wcpu_map.total_procs > wcpu_map.proc_cores) - ht_enabled = TRUE; - - /* Distribute the socket and core ids appropriately - * across the logical cores. For now, split the cores - * equally across the sockets. - */ - unsigned int lcore = 0; - for (unsigned int socket = 0; socket < - wcpu_map.proc_sockets; ++socket) { - for (unsigned int core = 0; - core < (wcpu_map.proc_cores / wcpu_map.proc_sockets); - ++core) { - wcpu_map.wlcore_map[lcore] - .socket_id = socket; - wcpu_map.wlcore_map[lcore] - .core_id = core; - lcore++; - if (ht_enabled) { - wcpu_map.wlcore_map[lcore] - .socket_id = socket; - wcpu_map.wlcore_map[lcore] - .core_id = core; - lcore++; + SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *infos, *info; + DWORD infos_size; + bool full = false; + + infos_size = 0; + if (!GetLogicalProcessorInformationEx( + RelationNumaNode, NULL, &infos_size)) { + DWORD error = GetLastError(); + if (error != ERROR_INSUFFICIENT_BUFFER) { + rte_panic("cannot get NUMA node info size, error %lu", + GetLastError()); + } + } + + infos = malloc(infos_size); + if (infos == NULL) { + rte_panic("cannot allocate memory for NUMA node information"); + return; + } + + if (!GetLogicalProcessorInformationEx( + RelationNumaNode, infos, &infos_size)) { + rte_panic("cannot get NUMA node information, error %lu", + GetLastError()); + } + + info = infos; + while ((uint8_t *)info - (uint8_t *)infos < infos_size) { + unsigned int node_id = info->NumaNode.NodeNumber; + GROUP_AFFINITY *cores = &info->NumaNode.GroupMask; + struct lcore_map *lcore; + unsigned int i, socket_id; + + /* NUMA node may be reported multiple times if it includes + * cores from different processor groups, e. g. 80 cores + * of a physical processor comprise one NUMA node, but two + * processor groups, because group size is limited by 32/64. + */ + for (socket_id = 0; socket_id < cpu_map.socket_count; + socket_id++) { + if (cpu_map.sockets[socket_id].node_id == node_id) + break; + } + + if (socket_id == cpu_map.socket_count) { + if (socket_id == RTE_DIM(cpu_map.sockets)) { + full = true; + goto exit; } + + cpu_map.sockets[socket_id].node_id = node_id; + cpu_map.socket_count++; + } + + for (i = 0; i < EAL_PROCESSOR_GROUP_SIZE; i++) { + if ((cores->Mask & ((KAFFINITY)1 << i)) == 0) + continue; + + if (cpu_map.lcore_count == RTE_DIM(cpu_map.lcores)) { + full = true; + goto exit; + } + + lcore = &cpu_map.lcores[cpu_map.lcore_count]; + lcore->socket_id = socket_id; + lcore->core_id = + cores->Group * EAL_PROCESSOR_GROUP_SIZE + i; + cpu_map.lcore_count++; } + + info = (SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *)( + (uint8_t *)info + info->Size); + } + +exit: + if (full) { + /* RTE_LOG() may not be available, but this is important. */ + fprintf(stderr, "Enumerated maximum of %u NUMA nodes and %u cores\n", + cpu_map.socket_count, cpu_map.lcore_count); } + + free(infos); } -/* - * Check if a cpu is present by the presence of the cpu information for it - */ int eal_cpu_detected(unsigned int lcore_id) { - return (lcore_id < wcpu_map.total_procs); + return lcore_id < cpu_map.lcore_count; } -/* - * Get CPU socket id for a logical core - */ unsigned eal_cpu_socket_id(unsigned int lcore_id) { - return wcpu_map.wlcore_map[lcore_id].socket_id; + return cpu_map.lcores[lcore_id].socket_id; } -/* - * Get CPU socket id (NUMA node) for a logical core - */ unsigned eal_cpu_core_id(unsigned int lcore_id) { - return wcpu_map.wlcore_map[lcore_id].core_id; + return cpu_map.lcores[lcore_id].core_id; +} + +unsigned int +eal_socket_numa_node(unsigned int socket_id) +{ + return cpu_map.sockets[socket_id].node_id; } diff --git a/lib/librte_eal/windows/eal_windows.h b/lib/librte_eal/windows/eal_windows.h index fadd676b2..390d2fd66 100644 --- a/lib/librte_eal/windows/eal_windows.h +++ b/lib/librte_eal/windows/eal_windows.h @@ -26,4 +26,14 @@ void eal_create_cpu_map(void); */ int eal_thread_create(pthread_t *thread); +/** + * Get system NUMA node number for a socket ID. + * + * @param socket_id + * Valid EAL socket ID. + * @return + * NUMA node number to use with Win32 API. + */ +unsigned int eal_socket_numa_node(unsigned int socket_id); + #endif /* _EAL_WINDOWS_H_ */ -- 2.25.1