From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 10C065F5D for ; Thu, 22 Mar 2018 06:16:16 +0100 (CET) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2M5Dk42044028 for ; Thu, 22 Mar 2018 01:16:16 -0400 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0b-001b2d01.pphosted.com with ESMTP id 2gv3t1vjer-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 22 Mar 2018 01:16:15 -0400 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 22 Mar 2018 05:16:14 -0000 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 22 Mar 2018 05:16:11 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w2M5GBaY12845286; Thu, 22 Mar 2018 05:16:11 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 813E24203F; Thu, 22 Mar 2018 05:08:17 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB26E42045; Thu, 22 Mar 2018 05:08:16 +0000 (GMT) Received: from [9.124.35.163] (unknown [9.124.35.163]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 22 Mar 2018 05:08:16 +0000 (GMT) To: "Burakov, Anatoly" Cc: dev@dpdk.org, Bruce Richardson , Chao Zhu References: <18deafea-5662-88ef-2ddc-3a1970d67405@linux.vnet.ibm.com> From: gowrishankar muthukrishnan Date: Thu, 22 Mar 2018 10:46:09 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-TM-AS-GCONF: 00 x-cbid: 18032205-0008-0000-0000-000004E0FFC6 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18032205-0009-0000-0000-00001E742595 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-22_03:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1803220061 Subject: Re: [dpdk-dev] [PATCH 18.05 v4] eal: add function to return number of detected sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2018 05:16:17 -0000 On Wednesday 21 March 2018 03:54 PM, Burakov, Anatoly wrote: > >>> +    config->numa_node_count = max_socket_id + 1; >> >> In some IBM servers, socket ID number does not seem to be in >> sequence. For an instance, 0 and 8 for a 2 node server. >> >> In this case, numa_node_count would mislead users if wrongly >> understood by its variable name IMO (see below) >>> +    RTE_LOG(INFO, EAL, "Detected %u NUMA nodes\n", >>> config->numa_node_count); >> >> For an instance, reading above message would tell 'EAL detected 8 >> nodes' in my server, but actually there are only two nodes. >> >> Could its name better be 'numa_node_id_max' ?. Also, we store in >> actual count of numa nodes in _count variable. >> >> Also, there could be a case when there is no local memory available >> to a numa node too. >> >> Thanks, >> Gowrishankar > > The point of this patchset is to (pre)allocate memory only on existing > sockets. > > If we don't know how many sockets there are, we are forced to > preallocate VA space per each *possible* NUMA node - that is, reserve > e.g. 8x128G of memory, 6 of which will go unused on a 2-socket system. > We can't know if there is no memory on socket in advance, but we can > at least avoid preallocating VA space for sockets that don't exist in > the first place. > Sounds good Anatoly. May be, sysfs/ might help to confirm if a numa node has local memory ?. Anyway, for the context of this particular patch (return numa nodes), below approach you mentioned is good. > How about we store all possible socket id's instead? e.g. something like: > > static int numa_node_ids[MAX_NUMA_NODES]; > <...> > int rte_eal_cpu_init() { >     int sockets[RTE_MAX_LCORE]; >     <...> >     for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { >         core_to_socket[lcore_id] = socket; sockets[lcore_id] = eal_cpu_socket_id(lcore_id); >     } >     <...> >     qsort(sockets); >     <...> >     // store all unique sockets in numa_node_ids in ascending order Just thinking that, is there a purpose of retaining a numa ID which does not have local memory attached ? but sockets[] is suppose to reflect all available nodes though (and assuming, its calling place to ensure for the existence of numa local memory). > } > <...> > > on a 2 socket system we then get: > > rte_num_sockets() => return 2 > rte_get_socket_id(int idx) => return numa_node_ids[idx] rte_get_socket_mem(idx) might help to validate for local memory existence ? > > Would that be suitable? > Thanks, Gowrishankar