From: "Varghese, Vipin" <Vipin.Varghese@amd.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "david.marchand@redhat.com" <david.marchand@redhat.com>,
"Tummala, Sivaprasad" <Sivaprasad.Tummala@amd.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: RE: [PATCH] usertools: enhance logic to display NUMA
Date: Mon, 28 Mar 2022 02:56:52 +0000 [thread overview]
Message-ID: <MN2PR12MB3085DB6773CBB668CCB0308E821D9@MN2PR12MB3085.namprd12.prod.outlook.com> (raw)
In-Reply-To: <3129842.AJdgDx1Vlc@thomas>
[AMD Official Use Only]
Hi Thomas,
<snipp>
26/03/2022 08:32, Vipin Varghese:
> enhance python logic to accomadate NUMA information. Current logic
> considers physical socket with CPU threads to core map. With new AMD
> SKU vairant NUMA is no longer same as SOCKET. Single physical socket
> can be partitioned to variant of 1,2 and 4.
>
> The changes address the new mapping with Socket-NUMA to CPU cores.
>
> Signed-off-by: Vipin Varghese <vipin.varghese@amd.com>
> ---
> usertools/cpu_layout.py | 76
> +++++++++++++++++++++++++----------------
> 1 file changed, 47 insertions(+), 29 deletions(-)
Honestly, I'm not sure it is a good idea to keep this script in the DPDK repo.
Can it be replaced with hwloc usage?
thanks for the suggestion, it is genuine and useful. Following is my observations
- It takes some effort to identify the NUMA with `Group`
- One needs to install ` lstopo-no-graphics` on distro and manually build and add on custom Linux.
What is the output on the new AMD SKU for this command?
lstopo-no-graphics --merge
I have tried ` lstopo-no-graphics --merge` on a ` 2 Socket AMD EPYC 7713 64-Core Processor` with
possible NUMA configuration such as 1, 2 and 4.
```
$ lstopo-no-graphics --merge
Machine (503GB total)
Package L#0
Group0 L#0
NUMANode L#0 (P#0 126GB)
L3 L#0 (32MB)
Core L#0
PU L#0 (P#0)
PU L#1 (P#128)
Core L#1
PU L#2 (P#1)
PU L#3 (P#129)
Core L#2
PU L#4 (P#2)
PU L#5 (P#130)
Core L#3
PU L#6 (P#3)
PU L#7 (P#131)
Core L#4
PU L#8 (P#4)
PU L#9 (P#132)
Core L#5
PU L#10 (P#5)
PU L#11 (P#133)
Core L#6
PU L#12 (P#6)
PU L#13 (P#134)
Core L#7
PU L#14 (P#7)
PU L#15 (P#135)
L3 L#1 (32MB)
Core L#8
PU L#16 (P#8)
PU L#17 (P#136)
Core L#9
PU L#18 (P#9)
PU L#19 (P#137)
Core L#10
PU L#20 (P#10)
PU L#21 (P#138)
Core L#11
PU L#22 (P#11)
PU L#23 (P#139)
Core L#12
PU L#24 (P#12)
PU L#25 (P#140)
Core L#13
PU L#26 (P#13)
PU L#27 (P#141)
Core L#14
PU L#28 (P#14)
PU L#29 (P#142)
Core L#15
PU L#30 (P#15)
PU L#31 (P#143)
L3 L#2 (32MB)
Core L#16
PU L#32 (P#16)
PU L#33 (P#144)
Core L#17
PU L#34 (P#17)
PU L#35 (P#145)
Core L#18
PU L#36 (P#18)
PU L#37 (P#146)
Core L#19
PU L#38 (P#19)
PU L#39 (P#147)
Core L#20
PU L#40 (P#20)
PU L#41 (P#148)
Core L#21
PU L#42 (P#21)
PU L#43 (P#149)
Core L#22
PU L#44 (P#22)
PU L#45 (P#150)
Core L#23
PU L#46 (P#23)
PU L#47 (P#151)
L3 L#3 (32MB)
Core L#24
PU L#48 (P#24)
PU L#49 (P#152)
Core L#25
PU L#50 (P#25)
PU L#51 (P#153)
Core L#26
PU L#52 (P#26)
PU L#53 (P#154)
Core L#27
PU L#54 (P#27)
PU L#55 (P#155)
Core L#28
PU L#56 (P#28)
PU L#57 (P#156)
Core L#29
PU L#58 (P#29)
PU L#59 (P#157)
Core L#30
PU L#60 (P#30)
PU L#61 (P#158)
Core L#31
PU L#62 (P#31)
PU L#63 (P#159)
HostBridge
PCIBridge
PCI 41:00.0 (Ethernet)
Net "ens1f0"
OpenFabrics "mlx5_0"
PCI 41:00.1 (Ethernet)
Net "ens1f1"
OpenFabrics "mlx5_1"
HostBridge
PCIBridge
PCI 63:00.0 (Ethernet)
Net "eno12399np0"
PCI 63:00.1 (Ethernet)
Net "eno12409np1"
PCIBridge
PCIBridge
PCI 62:00.0 (VGA)
Group0 L#1
NUMANode L#1 (P#1 126GB)
L3 L#4 (32MB)
Core L#32
PU L#64 (P#32)
PU L#65 (P#160)
Core L#33
PU L#66 (P#33)
PU L#67 (P#161)
Core L#34
PU L#68 (P#34)
PU L#69 (P#162)
Core L#35
PU L#70 (P#35)
PU L#71 (P#163)
Core L#36
PU L#72 (P#36)
PU L#73 (P#164)
Core L#37
PU L#74 (P#37)
PU L#75 (P#165)
Core L#38
PU L#76 (P#38)
PU L#77 (P#166)
Core L#39
PU L#78 (P#39)
PU L#79 (P#167)
L3 L#5 (32MB)
Core L#40
PU L#80 (P#40)
PU L#81 (P#168)
Core L#41
PU L#82 (P#41)
PU L#83 (P#169)
Core L#42
PU L#84 (P#42)
PU L#85 (P#170)
Core L#43
PU L#86 (P#43)
PU L#87 (P#171)
Core L#44
PU L#88 (P#44)
PU L#89 (P#172)
Core L#45
PU L#90 (P#45)
PU L#91 (P#173)
Core L#46
PU L#92 (P#46)
PU L#93 (P#174)
Core L#47
PU L#94 (P#47)
PU L#95 (P#175)
L3 L#6 (32MB)
Core L#48
PU L#96 (P#48)
PU L#97 (P#176)
Core L#49
PU L#98 (P#49)
PU L#99 (P#177)
Core L#50
PU L#100 (P#50)
PU L#101 (P#178)
Core L#51
PU L#102 (P#51)
PU L#103 (P#179)
Core L#52
PU L#104 (P#52)
PU L#105 (P#180)
Core L#53
PU L#106 (P#53)
PU L#107 (P#181)
Core L#54
PU L#108 (P#54)
PU L#109 (P#182)
Core L#55
PU L#110 (P#55)
PU L#111 (P#183)
L3 L#7 (32MB)
Core L#56
PU L#112 (P#56)
PU L#113 (P#184)
Core L#57
PU L#114 (P#57)
PU L#115 (P#185)
Core L#58
PU L#116 (P#58)
PU L#117 (P#186)
Core L#59
PU L#118 (P#59)
PU L#119 (P#187)
Core L#60
PU L#120 (P#60)
PU L#121 (P#188)
Core L#61
PU L#122 (P#61)
PU L#123 (P#189)
Core L#62
PU L#124 (P#62)
PU L#125 (P#190)
Core L#63
PU L#126 (P#63)
PU L#127 (P#191)
HostBridge
PCIBridge
PCI 01:00.0 (RAID)
PCIBridge
PCI 02:00.0 (SATA)
Block(Disk) "sda"
HostBridge
PCIBridge
PCI 22:00.0 (NVMExp)
Block(Disk) "nvme0n1"
Package L#1
Group0 L#2
NUMANode L#2 (P#2 126GB)
L3 L#8 (32MB)
Core L#64
PU L#128 (P#64)
PU L#129 (P#192)
Core L#65
PU L#130 (P#65)
PU L#131 (P#193)
Core L#66
PU L#132 (P#66)
PU L#133 (P#194)
Core L#67
PU L#134 (P#67)
PU L#135 (P#195)
Core L#68
PU L#136 (P#68)
PU L#137 (P#196)
Core L#69
PU L#138 (P#69)
PU L#139 (P#197)
Core L#70
PU L#140 (P#70)
PU L#141 (P#198)
Core L#71
PU L#142 (P#71)
PU L#143 (P#199)
L3 L#9 (32MB)
Core L#72
PU L#144 (P#72)
PU L#145 (P#200)
Core L#73
PU L#146 (P#73)
PU L#147 (P#201)
Core L#74
PU L#148 (P#74)
PU L#149 (P#202)
Core L#75
PU L#150 (P#75)
PU L#151 (P#203)
Core L#76
PU L#152 (P#76)
PU L#153 (P#204)
Core L#77
PU L#154 (P#77)
PU L#155 (P#205)
Core L#78
PU L#156 (P#78)
PU L#157 (P#206)
Core L#79
PU L#158 (P#79)
PU L#159 (P#207)
L3 L#10 (32MB)
Core L#80
PU L#160 (P#80)
PU L#161 (P#208)
Core L#81
PU L#162 (P#81)
PU L#163 (P#209)
Core L#82
PU L#164 (P#82)
PU L#165 (P#210)
Core L#83
PU L#166 (P#83)
PU L#167 (P#211)
Core L#84
PU L#168 (P#84)
PU L#169 (P#212)
Core L#85
PU L#170 (P#85)
PU L#171 (P#213)
Core L#86
PU L#172 (P#86)
PU L#173 (P#214)
Core L#87
PU L#174 (P#87)
PU L#175 (P#215)
L3 L#11 (32MB)
Core L#88
PU L#176 (P#88)
PU L#177 (P#216)
Core L#89
PU L#178 (P#89)
PU L#179 (P#217)
Core L#90
PU L#180 (P#90)
PU L#181 (P#218)
Core L#91
PU L#182 (P#91)
PU L#183 (P#219)
Core L#92
PU L#184 (P#92)
PU L#185 (P#220)
Core L#93
PU L#186 (P#93)
PU L#187 (P#221)
Core L#94
PU L#188 (P#94)
PU L#189 (P#222)
Core L#95
PU L#190 (P#95)
PU L#191 (P#223)
HostBridge
PCIBridge
PCI c5:00.0 (SATA)
HostBridge
PCIBridge
PCI e1:00.0 (Ethernet)
Net "eno8303"
PCI e1:00.1 (Ethernet)
Net "eno8403"
Group0 L#3
NUMANode L#3 (P#3 126GB)
L3 L#12 (32MB)
Core L#96
PU L#192 (P#96)
PU L#193 (P#224)
Core L#97
PU L#194 (P#97)
PU L#195 (P#225)
Core L#98
PU L#196 (P#98)
PU L#197 (P#226)
Core L#99
PU L#198 (P#99)
PU L#199 (P#227)
Core L#100
PU L#200 (P#100)
PU L#201 (P#228)
Core L#101
PU L#202 (P#101)
PU L#203 (P#229)
Core L#102
PU L#204 (P#102)
PU L#205 (P#230)
Core L#103
PU L#206 (P#103)
PU L#207 (P#231)
L3 L#13 (32MB)
Core L#104
PU L#208 (P#104)
PU L#209 (P#232)
Core L#105
PU L#210 (P#105)
PU L#211 (P#233)
Core L#106
PU L#212 (P#106)
PU L#213 (P#234)
Core L#107
PU L#214 (P#107)
PU L#215 (P#235)
Core L#108
PU L#216 (P#108)
PU L#217 (P#236)
Core L#109
PU L#218 (P#109)
PU L#219 (P#237)
Core L#110
PU L#220 (P#110)
PU L#221 (P#238)
Core L#111
PU L#222 (P#111)
PU L#223 (P#239)
L3 L#14 (32MB)
Core L#112
PU L#224 (P#112)
PU L#225 (P#240)
Core L#113
PU L#226 (P#113)
PU L#227 (P#241)
Core L#114
PU L#228 (P#114)
PU L#229 (P#242)
Core L#115
PU L#230 (P#115)
PU L#231 (P#243)
Core L#116
PU L#232 (P#116)
PU L#233 (P#244)
Core L#117
PU L#234 (P#117)
PU L#235 (P#245)
Core L#118
PU L#236 (P#118)
PU L#237 (P#246)
Core L#119
PU L#238 (P#119)
PU L#239 (P#247)
L3 L#15 (32MB)
Core L#120
PU L#240 (P#120)
PU L#241 (P#248)
Core L#121
PU L#242 (P#121)
PU L#243 (P#249)
Core L#122
PU L#244 (P#122)
PU L#245 (P#250)
Core L#123
PU L#246 (P#123)
PU L#247 (P#251)
Core L#124
PU L#248 (P#124)
PU L#249 (P#252)
Core L#125
PU L#250 (P#125)
PU L#251 (P#253)
Core L#126
PU L#252 (P#126)
PU L#253 (P#254)
Core L#127
PU L#254 (P#127)
PU L#255 (P#255)
HostBridge
PCIBridge
PCI 81:00.0 (Ethernet)
Net "ens3f0"
OpenFabrics "mlx5_2"
PCI 81:00.1 (Ethernet)
Net "ens3f1"
OpenFabrics "mlx5_3"
```
The current result gives
```
$ ./usertools/cpu_layout.py
======================================================================
Core and Socket Information (as reported by '/sys/devices/system/cpu')
======================================================================
cores = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]
numa nodes per socket = [0, 1]
sockets = [0, 1]
Socket 0 Numa 0
-------------------
Core 0 [0, 128]
Core 1 [1, 129]
Core 2 [2, 130]
Core 3 [3, 131]
Core 4 [4, 132]
Core 5 [5, 133]
Core 6 [6, 134]
Core 7 [7, 135]
Core 8 [8, 136]
Core 9 [9, 137]
Core 10 [10, 138]
Core 11 [11, 139]
Core 12 [12, 140]
Core 13 [13, 141]
Core 14 [14, 142]
Core 15 [15, 143]
Core 16 [16, 144]
Core 17 [17, 145]
Core 18 [18, 146]
Core 19 [19, 147]
Core 20 [20, 148]
Core 21 [21, 149]
Core 22 [22, 150]
Core 23 [23, 151]
Core 24 [24, 152]
Core 25 [25, 153]
Core 26 [26, 154]
Core 27 [27, 155]
Core 28 [28, 156]
Core 29 [29, 157]
Core 30 [30, 158]
Core 31 [31, 159]
Socket 0 Numa 1
-------------------
Core 0 [32, 160]
Core 1 [33, 161]
Core 2 [34, 162]
Core 3 [35, 163]
Core 4 [36, 164]
Core 5 [37, 165]
Core 6 [38, 166]
Core 7 [39, 167]
Core 8 [40, 168]
Core 9 [41, 169]
Core 10 [42, 170]
Core 11 [43, 171]
Core 12 [44, 172]
Core 13 [45, 173]
Core 14 [46, 174]
Core 15 [47, 175]
Core 16 [48, 176]
Core 17 [49, 177]
Core 18 [50, 178]
Core 19 [51, 179]
Core 20 [52, 180]
Core 21 [53, 181]
Core 22 [54, 182]
Core 23 [55, 183]
Core 24 [56, 184]
Core 25 [57, 185]
Core 26 [58, 186]
Core 27 [59, 187]
Core 28 [60, 188]
Core 29 [61, 189]
Core 30 [62, 190]
Core 31 [63, 191]
Socket 1 Numa 2
-------------------
Core 0 [64, 192]
Core 1 [65, 193]
Core 2 [66, 194]
Core 3 [67, 195]
Core 4 [68, 196]
Core 5 [69, 197]
Core 6 [70, 198]
Core 7 [71, 199]
Core 8 [72, 200]
Core 9 [73, 201]
Core 10 [74, 202]
Core 11 [75, 203]
Core 12 [76, 204]
Core 13 [77, 205]
Core 14 [78, 206]
Core 15 [79, 207]
Core 16 [80, 208]
Core 17 [81, 209]
Core 18 [82, 210]
Core 19 [83, 211]
Core 20 [84, 212]
Core 21 [85, 213]
Core 22 [86, 214]
Core 23 [87, 215]
Core 24 [88, 216]
Core 25 [89, 217]
Core 26 [90, 218]
Core 27 [91, 219]
Core 28 [92, 220]
Core 29 [93, 221]
Core 30 [94, 222]
Core 31 [95, 223]
Socket 1 Numa 3
-------------------
Core 0 [96, 224]
Core 1 [97, 225]
Core 2 [98, 226]
Core 3 [99, 227]
Core 4 [100, 228]
Core 5 [101, 229]
Core 6 [102, 230]
Core 7 [103, 231]
Core 8 [104, 232]
Core 9 [105, 233]
Core 10 [106, 234]
Core 11 [107, 235]
Core 12 [108, 236]
Core 13 [109, 237]
Core 14 [110, 238]
Core 15 [111, 239]
Core 16 [112, 240]
Core 17 [113, 241]
Core 18 [114, 242]
Core 19 [115, 243]
Core 20 [116, 244]
Core 21 [117, 245]
Core 22 [118, 246]
Core 23 [119, 247]
Core 24 [120, 248]
Core 25 [121, 249]
Core 26 [122, 250]
Core 27 [123, 251]
Core 28 [124, 252]
Core 29 [125, 253]
Core 30 [126, 254]
Core 31 [127, 255]
```
next prev parent reply other threads:[~2022-03-28 2:57 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-26 7:32 Vipin Varghese
2022-03-26 9:46 ` Thomas Monjalon
2022-03-28 2:56 ` Varghese, Vipin [this message]
2022-03-28 7:44 ` Tummala, Sivaprasad
2023-06-14 10:48 ` Ferruh Yigit
2023-06-14 11:25 ` Dmitry Kozlyuk
2023-06-14 14:30 ` Thomas Monjalon
2023-07-11 6:38 ` Varghese, Vipin
2023-07-11 15:42 ` Stephen Hemminger
2023-07-11 16:26 ` Thomas Monjalon
2023-07-14 9:14 ` Varghese, Vipin
2023-07-17 15:07 ` Thomas Monjalon
2023-07-18 10:37 ` Varghese, Vipin
2023-07-18 14:45 ` Thomas Monjalon
2023-08-12 11:39 ` Varghese, Vipin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MN2PR12MB3085DB6773CBB668CCB0308E821D9@MN2PR12MB3085.namprd12.prod.outlook.com \
--to=vipin.varghese@amd.com \
--cc=Sivaprasad.Tummala@amd.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).