From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D776A050B; Fri, 8 Apr 2022 16:01:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 36F33410FB; Fri, 8 Apr 2022 16:01:53 +0200 (CEST) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by mails.dpdk.org (Postfix) with ESMTP id 51B42410EC for ; Fri, 8 Apr 2022 16:01:52 +0200 (CEST) Received: by mail-lj1-f177.google.com with SMTP id a30so11545491ljq.13 for ; Fri, 08 Apr 2022 07:01:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hYWsLnvx/DzS7nN95YdS50Jq8J6AHRnPbr+hBowNmDU=; b=aAwWOFTm0TcV6vwiO3A2r3Zp2oTgcwSg4InxZn8RGW1LiTXlg1h5Imj0tk2R2lICMj /N9m7UsHVkU7/ZD/2UbgUjVVCjxbeH7n9bUaZqI6nR21D3jNKqNb1ybbcfeT2qoVO06/ mIYRczL3JXRqL9fYtKcyvykObZGjXYZusOQQsypt2UChDoSndVPbXUtxsiyNjsdSjtn5 cmMJUAwBFqAOQvOm490p1XaMT1oBFEcfUheCSx5+FgyI9XF5tMQsHkDff2Qj7cJ09/tn BEjtc70wvZWKbRmm0o+p8qAfUZah9CwbS+GaWJCPFUvpwEDSjDsCzAiPpJN8rg1Iptg3 zWzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hYWsLnvx/DzS7nN95YdS50Jq8J6AHRnPbr+hBowNmDU=; b=kX1z6gWzWBGMwVVOeU0xTdPwpH4VadXRT2ht2jnVg2MUHd7DWFMQdUKxDLUqOp9hj8 QmOj2oyYrN8WmetLHQktoLvZ1k+Untn4jyU6LxdQk9jLbjUFuKQ/GqKku0V+aEgDKnCv tv4RHoxWH5l7JHqSwPljag+tA9weIsJH0E8AEkj6ivRMOFDWij1OevSCkE4rNcIz9hJY +By7wHExnv3mgYHrNL55iyiQodjuGFpdWjQ9Apzu6juDAvxHCqnaK0G4p6qeRwXsVBxt pgUJPEcH5GxqGxo4ycIJu0inrOH/+9t5nR8wGHbtxEtv6zY958erwLWu3LzzpqgHqxZF xWrw== X-Gm-Message-State: AOAM5324umzQUTEzX88zkpvpSwlU8XFvUdC4VUg5ue3oZsOyhu1YPVuf XjSfzhysQedUXS5DvtwZebI= X-Google-Smtp-Source: ABdhPJxT50LV3YpsJ59Ihs0y1P4w4F1kDjk7up0ZjZqu24buOQRkG8Mk9R9AY+tXew4m7dU7wB1wDg== X-Received: by 2002:a2e:9dcf:0:b0:24b:4486:28ea with SMTP id x15-20020a2e9dcf000000b0024b448628eamr5225327ljj.410.1649426511775; Fri, 08 Apr 2022 07:01:51 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id t25-20020a2e9c59000000b0024af7b8fac2sm2248315ljj.6.2022.04.08.07.01.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Apr 2022 07:01:51 -0700 (PDT) Date: Fri, 8 Apr 2022 17:01:50 +0300 From: Dmitry Kozlyuk To: Tyler Retzlaff Cc: dev@dpdk.org, thomas@monjalon.net, anatoly.burakov@intel.com, Narcisa Vasile Subject: Re: [PATCH 2/3] eal: implement functions for get/set thread affinity Message-ID: <20220408170150.794bdbc4@sovereign> In-Reply-To: <1648819793-18948-3-git-send-email-roretzla@linux.microsoft.com> References: <1648819793-18948-1-git-send-email-roretzla@linux.microsoft.com> <1648819793-18948-3-git-send-email-roretzla@linux.microsoft.com> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 2022-04-01 06:29 (UTC-0700), Tyler Retzlaff: > Implement functions for getting/setting thread affinity. > Threads can be pinned to specific cores by setting their > affinity attribute. > > Signed-off-by: Narcisa Vasile > Signed-off-by: Tyler Retzlaff Acked-by: Dmitry Kozlyuk Please see some small comments below. > --- > lib/eal/include/rte_thread.h | 45 ++++++++++ > lib/eal/unix/rte_thread.c | 16 ++++ > lib/eal/version.map | 4 + > lib/eal/windows/eal_lcore.c | 173 +++++++++++++++++++++++++++++---------- > lib/eal/windows/eal_windows.h | 10 +++ > lib/eal/windows/include/rte_os.h | 2 + > lib/eal/windows/rte_thread.c | 131 ++++++++++++++++++++++++++++- > 7 files changed, 336 insertions(+), 45 deletions(-) > > diff --git a/lib/eal/include/rte_thread.h b/lib/eal/include/rte_thread.h > index 8be8ed8..4eb113f 100644 > --- a/lib/eal/include/rte_thread.h > +++ b/lib/eal/include/rte_thread.h > @@ -2,6 +2,8 @@ > * Copyright(c) 2021 Mellanox Technologies, Ltd > */ > > +#include > + > #include > #include > > @@ -21,6 +23,13 @@ > #endif > > /** > + * Thread id descriptor. > + */ > +typedef struct rte_thread_tag { > + uintptr_t opaque_id; /**< thread identifier */ > +} rte_thread_t; > + > +/** > * TLS key type, an opaque pointer. > */ > typedef struct eal_tls_key *rte_thread_key; > @@ -28,6 +37,42 @@ > #ifdef RTE_HAS_CPUSET > > /** Missing a common part for experimental functions: * @warning * @b EXPERIMENTAL: this API may change without prior notice. > + * Set the affinity of thread 'thread_id' to the cpu set > + * specified by 'cpuset'. > + * > + * @param thread_id > + * Id of the thread for which to set the affinity. > + * > + * @param cpuset > + * Pointer to CPU affinity to set. > + * > + * @return > + * On success, return 0. > + * On failure, return a positive errno-style error number. > + */ > +__rte_experimental > +int rte_thread_set_affinity_by_id(rte_thread_t thread_id, > + const rte_cpuset_t *cpuset); > + > +/** Same here. > + * Get the affinity of thread 'thread_id' and store it > + * in 'cpuset'. > + * > + * @param thread_id > + * Id of the thread for which to get the affinity. > + * > + * @param cpuset > + * Pointer for storing the affinity value. > + * > + * @return > + * On success, return 0. > + * On failure, return a positive errno-style error number. > + */ > +__rte_experimental > +int rte_thread_get_affinity_by_id(rte_thread_t thread_id, > + rte_cpuset_t *cpuset); > + > +/** > * Set core affinity of the current thread. > * Support both EAL and non-EAL thread and update TLS. > * [...] > +static int > +eal_query_group_affinity(void) > +{ > + SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *infos = NULL; > + unsigned int *cpu_count = &cpu_map.cpu_count; > + DWORD infos_size = 0; > + int ret = 0; > + USHORT group_count; > + KAFFINITY affinity; > + USHORT group_no; > + unsigned int i; > + > + if (!GetLogicalProcessorInformationEx(RelationGroup, NULL, > + &infos_size)) { > + DWORD error = GetLastError(); > + if (error != ERROR_INSUFFICIENT_BUFFER) { > + log_early("Cannot get group information size, " > + "error %lu\n", error); Please don't break string constants for easy search. > + rte_errno = EINVAL; > + ret = -1; > + goto cleanup; > + } > + } [...] > +static bool > +eal_create_lcore_map(const SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX *info) > +{ > + const unsigned int node_id = info->NumaNode.NodeNumber; > + const GROUP_AFFINITY *cores = &info->NumaNode.GroupMask; > + struct lcore_map *lcore; > + unsigned int socket_id; > + unsigned int i; > + > + /* NUMA node may be reported multiple times if it includes > + * cores from different processor groups, e. g. 80 cores > + * of a physical processor comprise one NUMA node, but two > + * processor groups, because group size is limited by 32/64. > + */ > + for (socket_id = 0; socket_id < cpu_map.socket_count; socket_id++) { > + if (cpu_map.sockets[socket_id].node_id == node_id) > + break; > + } Nit: multi-line comments should start with a line containing just "/*", and {} are no needed here. [...] > +static int > +rte_convert_cpuset_to_affinity(const rte_cpuset_t *cpuset, > + PGROUP_AFFINITY affinity) > +{ > + int ret = 0; > + PGROUP_AFFINITY cpu_affinity = NULL; > + unsigned int cpu_idx; > + > + memset(affinity, 0, sizeof(GROUP_AFFINITY)); > + affinity->Group = (USHORT)-1; > + > + /* Check that all cpus of the set belong to the same processor group and > + * accumulate thread affinity to be applied. > + */ > + for (cpu_idx = 0; cpu_idx < CPU_SETSIZE; cpu_idx++) { > + if (!CPU_ISSET(cpu_idx, cpuset)) > + continue; > + > + cpu_affinity = eal_get_cpu_affinity(cpu_idx); > + > + if (affinity->Group == (USHORT)-1) { > + affinity->Group = cpu_affinity->Group; > + } else if (affinity->Group != cpu_affinity->Group) { > + ret = EINVAL; Should it be ENOTSUP, because we don't support this case? This limitation is also worth documenting in the commit log. > + goto cleanup; > + } > + > + affinity->Mask |= cpu_affinity->Mask; > + } > + > + if (affinity->Mask == 0) { > + ret = EINVAL; > + goto cleanup; > + } > + > +cleanup: > + return ret; > +} [...]