From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <gowrishankar.m@linux.vnet.ibm.com>
Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com
[148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 17DEA5F72
for <dev@dpdk.org>; Wed, 21 Mar 2018 06:00:02 +0100 (CET)
Received: from pps.filterd (m0098416.ppops.net [127.0.0.1])
by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id
w2L4sPvP178492 for <dev@dpdk.org>; Wed, 21 Mar 2018 01:00:01 -0400
Received: from e06smtp15.uk.ibm.com (e06smtp15.uk.ibm.com [195.75.94.111])
by mx0b-001b2d01.pphosted.com with ESMTP id 2guf9j2n0h-1
(version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT)
for <dev@dpdk.org>; Wed, 21 Mar 2018 01:00:01 -0400
Received: from localhost
by e06smtp15.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only!
Violators will be prosecuted
for <dev@dpdk.org> from <gowrishankar.m@linux.vnet.ibm.com>;
Wed, 21 Mar 2018 04:59:59 -0000
Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194)
by e06smtp15.uk.ibm.com (192.168.101.145) with IBM ESMTP SMTP Gateway:
Authorized Use Only! Violators will be prosecuted;
Wed, 21 Mar 2018 04:59:56 -0000
Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com
[9.149.105.59])
by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id
w2L4xuXp1179954; Wed, 21 Mar 2018 04:59:56 GMT
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 67481A4053;
Wed, 21 Mar 2018 04:52:39 +0000 (GMT)
Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 94B7CA404D;
Wed, 21 Mar 2018 04:52:38 +0000 (GMT)
Received: from [9.124.35.231] (unknown [9.124.35.231])
by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP;
Wed, 21 Mar 2018 04:52:38 +0000 (GMT)
To: Anatoly Burakov <anatoly.burakov@intel.com>
Cc: dev@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>,
Chao Zhu <chaozhu@linux.vnet.ibm.com>
References: <cover.1517925320.git.anatoly.burakov@intel.com>
<eb142226e1e62abe5a95bb135468732131580b71.1517925320.git.anatoly.burakov@intel.com>
From: gowrishankar muthukrishnan <gowrishankar.m@linux.vnet.ibm.com>
Date: Wed, 21 Mar 2018 10:29:54 +0530
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <eb142226e1e62abe5a95bb135468732131580b71.1517925320.git.anatoly.burakov@intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-TM-AS-GCONF: 00
x-cbid: 18032104-0020-0000-0000-0000040798C6
X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused
x-cbparentid: 18032104-0021-0000-0000-0000429BB542
Message-Id: <18deafea-5662-88ef-2ddc-3a1970d67405@linux.vnet.ibm.com>
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, ,
definitions=2018-03-21_01:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
priorityscore=1501
malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0
clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000
definitions=main-1803210062
Subject: Re: [dpdk-dev] [PATCH 18.05 v4] eal: add function to return number
of detected sockets
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Mar 2018 05:00:02 -0000
On Wednesday 07 February 2018 03:28 PM, Anatoly Burakov wrote:
> During lcore scan, find maximum socket ID and store it. This will
> break the ABI, so bump ABI version.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>
> Notes:
> v4:
> - Remove backwards ABI compatibility, bump ABI instead
>
> v3:
> - Added ABI compatibility
>
> v2:
> - checkpatch changes
> - check socket before deciding if the core is not to be used
>
> lib/librte_eal/bsdapp/eal/Makefile | 2 +-
> lib/librte_eal/common/eal_common_lcore.c | 37 +++++++++++++++++++++----------
> lib/librte_eal/common/include/rte_eal.h | 1 +
> lib/librte_eal/common/include/rte_lcore.h | 8 +++++++
> lib/librte_eal/linuxapp/eal/Makefile | 2 +-
> lib/librte_eal/rte_eal_version.map | 9 +++++++-
> 6 files changed, 44 insertions(+), 15 deletions(-)
>
> diff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile
> index dd455e6..ed1d17b 100644
> --- a/lib/librte_eal/bsdapp/eal/Makefile
> +++ b/lib/librte_eal/bsdapp/eal/Makefile
> @@ -21,7 +21,7 @@ LDLIBS += -lgcc_s
>
> EXPORT_MAP := ../../rte_eal_version.map
>
> -LIBABIVER := 6
> +LIBABIVER := 7
>
> # specific to bsdapp exec-env
> SRCS-$(CONFIG_RTE_EXEC_ENV_BSDAPP) := eal.c
> diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c
> index 7724fa4..827ddeb 100644
> --- a/lib/librte_eal/common/eal_common_lcore.c
> +++ b/lib/librte_eal/common/eal_common_lcore.c
> @@ -28,6 +28,7 @@ rte_eal_cpu_init(void)
> struct rte_config *config = rte_eal_get_configuration();
> unsigned lcore_id;
> unsigned count = 0;
> + unsigned int socket_id, max_socket_id = 0;
>
> /*
> * Parse the maximum set of logical cores, detect the subset of running
> @@ -39,6 +40,19 @@ rte_eal_cpu_init(void)
> /* init cpuset for per lcore config */
> CPU_ZERO(&lcore_config[lcore_id].cpuset);
>
> + /* find socket first */
> + socket_id = eal_cpu_socket_id(lcore_id);
> + if (socket_id >= RTE_MAX_NUMA_NODES) {
> +#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
> + socket_id = 0;
> +#else
> + RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than RTE_MAX_NUMA_NODES (%d)\n",
> + socket_id, RTE_MAX_NUMA_NODES);
> + return -1;
> +#endif
> + }
> + max_socket_id = RTE_MAX(max_socket_id, socket_id);
> +
> /* in 1:1 mapping, record related cpu detected state */
> lcore_config[lcore_id].detected = eal_cpu_detected(lcore_id);
> if (lcore_config[lcore_id].detected == 0) {
> @@ -54,18 +68,7 @@ rte_eal_cpu_init(void)
> config->lcore_role[lcore_id] = ROLE_RTE;
> lcore_config[lcore_id].core_role = ROLE_RTE;
> lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
> - lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
> - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
> -#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
> - lcore_config[lcore_id].socket_id = 0;
> -#else
> - RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
> - "RTE_MAX_NUMA_NODES (%d)\n",
> - lcore_config[lcore_id].socket_id,
> - RTE_MAX_NUMA_NODES);
> - return -1;
> -#endif
> - }
> + lcore_config[lcore_id].socket_id = socket_id;
> RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
> "core %u on socket %u\n",
> lcore_id, lcore_config[lcore_id].core_id,
> @@ -79,5 +82,15 @@ rte_eal_cpu_init(void)
> RTE_MAX_LCORE);
> RTE_LOG(INFO, EAL, "Detected %u lcore(s)\n", config->lcore_count);
>
> + config->numa_node_count = max_socket_id + 1;
In some IBM servers, socket ID number does not seem to be in sequence.
For an instance, 0 and 8 for a 2 node server.
In this case, numa_node_count would mislead users if wrongly understood
by its variable name IMO (see below)
> + RTE_LOG(INFO, EAL, "Detected %u NUMA nodes\n", config->numa_node_count);
For an instance, reading above message would tell 'EAL detected 8 nodes'
in my server, but actually there are only two nodes.
Could its name better be 'numa_node_id_max' ?. Also, we store in actual
count of numa nodes in _count variable.
Also, there could be a case when there is no local memory available to a
numa node too.
Thanks,
Gowrishankar
> +
> return 0;
> }
> +
> +unsigned int
> +rte_num_sockets(void)
> +{
> + const struct rte_config *config = rte_eal_get_configuration();
> + return config->numa_node_count;
> +}
> diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h
> index 08c6637..63fcc2e 100644
> --- a/lib/librte_eal/common/include/rte_eal.h
> +++ b/lib/librte_eal/common/include/rte_eal.h
> @@ -57,6 +57,7 @@ enum rte_proc_type_t {
> struct rte_config {
> uint32_t master_lcore; /**< Id of the master lcore */
> uint32_t lcore_count; /**< Number of available logical cores. */
> + uint32_t numa_node_count; /**< Number of detected NUMA nodes. */
> uint32_t service_lcore_count;/**< Number of available service cores. */
> enum rte_lcore_role_t lcore_role[RTE_MAX_LCORE]; /**< State of cores. */
>
> diff --git a/lib/librte_eal/common/include/rte_lcore.h b/lib/librte_eal/common/include/rte_lcore.h
> index d84bcff..ddf4c64 100644
> --- a/lib/librte_eal/common/include/rte_lcore.h
> +++ b/lib/librte_eal/common/include/rte_lcore.h
> @@ -120,6 +120,14 @@ rte_lcore_index(int lcore_id)
> unsigned rte_socket_id(void);
>
> /**
> + * Return number of physical sockets on the system.
> + * @return
> + * the number of physical sockets as recognized by EAL
> + *
> + */
> +unsigned int rte_num_sockets(void);
> +
> +/**
> * Get the ID of the physical socket of the specified lcore
> *
> * @param lcore_id
> diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile
> index 7e5bbe8..b9c7727 100644
> --- a/lib/librte_eal/linuxapp/eal/Makefile
> +++ b/lib/librte_eal/linuxapp/eal/Makefile
> @@ -10,7 +10,7 @@ ARCH_DIR ?= $(RTE_ARCH)
> EXPORT_MAP := ../../rte_eal_version.map
> VPATH += $(RTE_SDK)/lib/librte_eal/common/arch/$(ARCH_DIR)
>
> -LIBABIVER := 6
> +LIBABIVER := 7
>
> VPATH += $(RTE_SDK)/lib/librte_eal/common
>
> diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
> index 4146907..fc83e74 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -211,6 +211,13 @@ DPDK_18.02 {
>
> } DPDK_17.11;
>
> +DPDK_18.05 {
> + global:
> +
> + rte_num_sockets;
> +
> +} DPDK_18.02;
> +
> EXPERIMENTAL {
> global:
>
> @@ -255,4 +262,4 @@ EXPERIMENTAL {
> rte_service_set_stats_enable;
> rte_service_start_with_defaults;
>
> -} DPDK_18.02;
> +} DPDK_18.05;