From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from e23smtp07.au.ibm.com (e23smtp07.au.ibm.com [202.81.31.140]) by dpdk.org (Postfix) with ESMTP id 48BDDAD9F for ; Fri, 20 May 2016 05:03:36 +0200 (CEST) Received: from localhost by e23smtp07.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 20 May 2016 13:03:34 +1000 Received: from d23dlp03.au.ibm.com (202.81.31.214) by e23smtp07.au.ibm.com (202.81.31.204) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 20 May 2016 13:03:31 +1000 X-IBM-Helo: d23dlp03.au.ibm.com X-IBM-MailFrom: chaozhu@linux.vnet.ibm.com X-IBM-RcptTo: dev@dpdk.org Received: from d23relay08.au.ibm.com (d23relay08.au.ibm.com [9.185.71.33]) by d23dlp03.au.ibm.com (Postfix) with ESMTP id 215A73578056 for ; Fri, 20 May 2016 13:03:31 +1000 (EST) Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u4K33Nh040370302 for ; Fri, 20 May 2016 13:03:31 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u4K32w8s000577 for ; Fri, 20 May 2016 13:02:58 +1000 Received: from ADMINIB2M8Q79C ([9.186.50.139]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u4K32sBS031811; Fri, 20 May 2016 13:02:56 +1000 From: "Chao Zhu" To: "'Bruce Richardson'" , "'Sergio Gonzalez Monroy'" Cc: "'Gowrishankar'" , , "'David Marchand'" References: <1457360003-30055-1-git-send-email-gowrishankar.m@linux.vnet.ibm.com> <56F17454.3010907@intel.com> <20160322171046.GG20448@bricha3-MOBL3> In-Reply-To: <20160322171046.GG20448@bricha3-MOBL3> Date: Fri, 20 May 2016 11:03:34 +0800 Message-ID: <000101d1b244$4d74de40$e85e9ac0$@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQKpaLNkd1e//xDx7eopFpyyS8yKkQHZs50XAggpvNWd8qLeMA== Content-Language: zh-cn X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16052003-0025-0000-0000-000004BF1238 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused Subject: Re: [dpdk-dev] [PATCH] eal/ppc: fix secondary process to map hugepages in correct order X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2016 03:03:38 -0000 Bruce, Recently, we find some bugs with mmap in PowerLinux. The mmap doesn't respect the address hints. In function get_virtual_area() in = eal_memory.c, mmap get the free virtual address range as the address hint. However, = when mapping the real memory in rte_eal_hugepage_init(), mmap doesn't return = the same address as the requested address. When taking a look at the /proc//maps, the requested address range is free for use. With this bug, pre-allocate some free space doesn't work. We're trying to create some test case and report it as a bug to kernel community. Here's some logs: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D EAL: Ask a virtual area of 0x10000000 bytes EAL: Virtual area found at 0x3fffa7000000 (size =3D 0x10000000) EAL: map_all_hugepages, /mnt/huge/rtemap_52,paddr 0x3ca6000000 = requested addr: 0x3fffa7000000 mmaped addr: 0x3efff0000000 EAL: map_all_hugepages, /mnt/huge/rtemap_53,paddr 0x3ca5000000 = requested addr: 0x3fffa8000000 mmaped addr: 0x3effef000000 EAL: map_all_hugepages, /mnt/huge/rtemap_54,paddr 0x3ca4000000 = requested addr: 0x3fffa9000000 mmaped addr: 0x3effee000000 EAL: map_all_hugepages, /mnt/huge/rtemap_55,paddr 0x3ca3000000 = requested addr: 0x3fffaa000000 mmaped addr: 0x3effed000000 EAL: map_all_hugepages, /mnt/huge/rtemap_56,paddr 0x3ca2000000 = requested addr: 0x3fffab000000 mmaped addr: 0x3effec000000 EAL: map_all_hugepages, /mnt/huge/rtemap_57,paddr 0x3ca1000000 = requested addr: 0x3fffac000000 mmaped addr: 0x3effeb000000 EAL: map_all_hugepages, /mnt/huge/rtemap_58,paddr 0x3ca0000000 = requested addr: 0x3fffad000000 mmaped addr: 0x3effea000000 EAL: map_all_hugepages, /mnt/huge/rtemap_59,paddr 0x3c9f000000 = requested addr: 0x3fffae000000 mmaped addr: 0x3effe9000000 EAL: map_all_hugepages, /mnt/huge/rtemap_60,paddr 0x3c9e000000 = requested addr: 0x3fffaf000000 mmaped addr: 0x3effe8000000 EAL: map_all_hugepages, /mnt/huge/rtemap_61,paddr 0x3c9d000000 = requested addr: 0x3fffb0000000 mmaped addr: 0x3effe7000000 EAL: map_all_hugepages, /mnt/huge/rtemap_62, paddr 0x3c9c000000 = requested addr: 0x3fffb1000000 mmaped addr: 0x3effe6000000 EAL: map_all_hugepages, /mnt/huge/rtemap_63, paddr 0x3c9b000000 = requested addr: 0x3fffb2000000 mmaped addr: 0x3effe5000000 EAL: map_all_hugepages, /mnt/huge/rtemap_51, paddr 0x3c9a000000 = requested addr: 0x3fffb3000000 mmaped addr: 0x3effe4000000 EAL: map_all_hugepages, /mnt/huge/rtemap_50, paddr 0x3c99000000 = requested addr: 0x3fffb4000000 mmaped addr: 0x3effe3000000 EAL: map_all_hugepages, /mnt/huge/rtemap_49, paddr 0x3c98000000 = requested addr: 0x3fffb5000000 mmaped addr: 0x3effe2000000 EAL: map_all_hugepages, /mnt/huge/rtemap_48, paddr 0x3c97000000 = requested addr: 0x3fffb6000000 mmaped addr: 0x3effe1000000 # cat /proc/143765/maps 01000000-02000000 rw-s 00000000 00:27 61162550 /mnt/huge/rtemap_14 02000000-03000000 rw-s 00000000 00:27 61162536 /mnt/huge/rtemap_0 03000000-04000000 rw-s 00000000 00:27 61162537 /mnt/huge/rtemap_1 04000000-05000000 rw-s 00000000 00:27 61162538 /mnt/huge/rtemap_2 05000000-06000000 rw-s 00000000 00:27 61162539 /mnt/huge/rtemap_3 06000000-07000000 rw-s 00000000 00:27 61162540 /mnt/huge/rtemap_4 07000000-08000000 rw-s 00000000 00:27 61162541 /mnt/huge/rtemap_5 08000000-09000000 rw-s 00000000 00:27 61162542 /mnt/huge/rtemap_6 09000000-0a000000 rw-s 00000000 00:27 61162543 /mnt/huge/rtemap_7 0a000000-0b000000 rw-s 00000000 00:27 61162544 /mnt/huge/rtemap_8 0b000000-0c000000 rw-s 00000000 00:27 61162545 /mnt/huge/rtemap_9 0c000000-0d000000 rw-s 00000000 00:27 61162546 /mnt/huge/rtemap_10 0d000000-0e000000 rw-s 00000000 00:27 61162547 /mnt/huge/rtemap_11 0e000000-0f000000 rw-s 00000000 00:27 61162548 /mnt/huge/rtemap_12 0f000000-10000000 rw-s 00000000 00:27 61162549 /mnt/huge/rtemap_13 10000000-101f0000 r-xp 00000000 08:32 6040458 /home/dpdk/build/app/test 101f0000-10220000 rw-p 001f0000 08:32 6040458 /home/dpdk/build/app/test 10220000-15c20000 rw-p 00000000 00:00 0 [heap] 20000000-21000000 rw-s 00000000 00:27 61162566 /mnt/huge/rtemap_30 21000000-22000000 rw-s 00000000 00:27 61162567 /mnt/huge/rtemap_31 22000000-23000000 rw-s 00000000 00:27 61162568 /mnt/huge/rtemap_32 23000000-24000000 rw-s 00000000 00:27 61162569 /mnt/huge/rtemap_33 24000000-25000000 rw-s 00000000 00:27 61162570 /mnt/huge/rtemap_34 25000000-26000000 rw-s 00000000 00:27 61162571 /mnt/huge/rtemap_35 26000000-27000000 rw-s 00000000 00:27 61162572 /mnt/huge/rtemap_36 27000000-28000000 rw-s 00000000 00:27 61162573 /mnt/huge/rtemap_37 28000000-29000000 rw-s 00000000 00:27 61162574 /mnt/huge/rtemap_38 29000000-2a000000 rw-s 00000000 00:27 61162575 /mnt/huge/rtemap_39 2a000000-2b000000 rw-s 00000000 00:27 61162576 /mnt/huge/rtemap_40 2b000000-2c000000 rw-s 00000000 00:27 61162577 /mnt/huge/rtemap_41 2c000000-2d000000 rw-s 00000000 00:27 61162578 /mnt/huge/rtemap_42 2d000000-2e000000 rw-s 00000000 00:27 61162579 /mnt/huge/rtemap_43 2e000000-2f000000 rw-s 00000000 00:27 61162580 /mnt/huge/rtemap_44 2f000000-30000000 rw-s 00000000 00:27 61162581 /mnt/huge/rtemap_45 30000000-31000000 rw-s 00000000 00:27 61162582 /mnt/huge/rtemap_46 31000000-32000000 rw-s 00000000 00:27 61162583 /mnt/huge/rtemap_47 32000000-33000000 rw-s 00000000 00:27 61162584 /mnt/huge/rtemap_48 33000000-34000000 rw-s 00000000 00:27 61162585 /mnt/huge/rtemap_49 34000000-35000000 rw-s 00000000 00:27 61162586 /mnt/huge/rtemap_50 35000000-36000000 rw-s 00000000 00:27 61162587 /mnt/huge/rtemap_51 36000000-37000000 rw-s 00000000 00:27 61162588 /mnt/huge/rtemap_52 37000000-38000000 rw-s 00000000 00:27 61162589 /mnt/huge/rtemap_53 38000000-39000000 rw-s 00000000 00:27 61162590 /mnt/huge/rtemap_54 39000000-3a000000 rw-s 00000000 00:27 61162591 /mnt/huge/rtemap_55 3a000000-3b000000 rw-s 00000000 00:27 61162592 /mnt/huge/rtemap_56 3b000000-3c000000 rw-s 00000000 00:27 61162593 /mnt/huge/rtemap_57 3c000000-3d000000 rw-s 00000000 00:27 61162594 /mnt/huge/rtemap_58 3d000000-3e000000 rw-s 00000000 00:27 61162595 /mnt/huge/rtemap_59 3e000000-3f000000 rw-s 00000000 00:27 61162596 /mnt/huge/rtemap_60 3f000000-40000000 rw-s 00000000 00:27 61162597 /mnt/huge/rtemap_61 40000000-41000000 rw-s 00000000 00:27 61162598 /mnt/huge/rtemap_62 41000000-42000000 rw-s 00000000 00:27 61162599 /mnt/huge/rtemap_63 3effb1000000-3effb2000000 rw-s 00000000 00:27 61162541 /mnt/huge/rtemap_5 3effb2000000-3effb3000000 rw-s 00000000 00:27 61162540 /mnt/huge/rtemap_4 3effb3000000-3effb4000000 rw-s 00000000 00:27 61162551 /mnt/huge/rtemap_15 3effb4000000-3effb5000000 rw-s 00000000 00:27 61162538 /mnt/huge/rtemap_2 3effb5000000-3effb6000000 rw-s 00000000 00:27 61162549 /mnt/huge/rtemap_13 3effb6000000-3effb7000000 rw-s 00000000 00:27 61162544 /mnt/huge/rtemap_8 3effb7000000-3effb8000000 rw-s 00000000 00:27 61162543 /mnt/huge/rtemap_7 3effb8000000-3effb9000000 rw-s 00000000 00:27 61162548 /mnt/huge/rtemap_12 3effb9000000-3effba000000 rw-s 00000000 00:27 61162537 /mnt/huge/rtemap_1 3effba000000-3effbb000000 rw-s 00000000 00:27 61162550 /mnt/huge/rtemap_14 3effbb000000-3effbc000000 rw-s 00000000 00:27 61162545 /mnt/huge/rtemap_9 3effbc000000-3effbd000000 rw-s 00000000 00:27 61162546 /mnt/huge/rtemap_10 3effbd000000-3effbe000000 rw-s 00000000 00:27 61162547 /mnt/huge/rtemap_11 3effbe000000-3effbf000000 rw-s 00000000 00:27 61162539 /mnt/huge/rtemap_3 3effbf000000-3effc0000000 rw-s 00000000 00:27 61162542 /mnt/huge/rtemap_6 3effc0000000-3effc1000000 rw-s 00000000 00:27 61162536 /mnt/huge/rtemap_0 3effc1000000-3effc2000000 rw-s 00000000 00:27 61162556 /mnt/huge/rtemap_20 3effc2000000-3effc3000000 rw-s 00000000 00:27 61162552 /mnt/huge/rtemap_16 3effc3000000-3effc4000000 rw-s 00000000 00:27 61162553 /mnt/huge/rtemap_17 3effc4000000-3effc5000000 rw-s 00000000 00:27 61162554 /mnt/huge/rtemap_18 3effc5000000-3effc6000000 rw-s 00000000 00:27 61162555 /mnt/huge/rtemap_19 3effc6000000-3effc7000000 rw-s 00000000 00:27 61162567 /mnt/huge/rtemap_31 3effc7000000-3effc8000000 rw-s 00000000 00:27 61162566 /mnt/huge/rtemap_30 3effc8000000-3effc9000000 rw-s 00000000 00:27 61162558 /mnt/huge/rtemap_22 3effc9000000-3effca000000 rw-s 00000000 00:27 61162557 /mnt/huge/rtemap_21 3effca000000-3effcb000000 rw-s 00000000 00:27 61162560 /mnt/huge/rtemap_24 3effcb000000-3effcc000000 rw-s 00000000 00:27 61162561 /mnt/huge/rtemap_25 3effcc000000-3effcd000000 rw-s 00000000 00:27 61162564 /mnt/huge/rtemap_28 3effcd000000-3effce000000 rw-s 00000000 00:27 61162559 /mnt/huge/rtemap_23 3effce000000-3effcf000000 rw-s 00000000 00:27 61162563 /mnt/huge/rtemap_27 3effcf000000-3effd0000000 rw-s 00000000 00:27 61162562 /mnt/huge/rtemap_26 3effd0000000-3effd1000000 rw-s 00000000 00:27 61162565 /mnt/huge/rtemap_29 3effd1000000-3effd2000000 rw-s 00000000 00:27 61162572 /mnt/huge/rtemap_36 3effd2000000-3effd3000000 rw-s 00000000 00:27 61162568 /mnt/huge/rtemap_32 3effd3000000-3effd4000000 rw-s 00000000 00:27 61162569 /mnt/huge/rtemap_33 3effd4000000-3effd5000000 rw-s 00000000 00:27 61162570 /mnt/huge/rtemap_34 3effd5000000-3effd6000000 rw-s 00000000 00:27 61162571 /mnt/huge/rtemap_35 3effd6000000-3effd7000000 rw-s 00000000 00:27 61162583 /mnt/huge/rtemap_47 3effd7000000-3effd8000000 rw-s 00000000 00:27 61162582 /mnt/huge/rtemap_46 3effd8000000-3effd9000000 rw-s 00000000 00:27 61162581 /mnt/huge/rtemap_45 3effd9000000-3effda000000 rw-s 00000000 00:27 61162580 /mnt/huge/rtemap_44 3effda000000-3effdb000000 rw-s 00000000 00:27 61162579 /mnt/huge/rtemap_43 3effdb000000-3effdc000000 rw-s 00000000 00:27 61162578 /mnt/huge/rtemap_42 3effdc000000-3effdd000000 rw-s 00000000 00:27 61162577 /mnt/huge/rtemap_41 3effdd000000-3effde000000 rw-s 00000000 00:27 61162574 /mnt/huge/rtemap_38 3effde000000-3effdf000000 rw-s 00000000 00:27 61162573 /mnt/huge/rtemap_37 3effdf000000-3effe0000000 rw-s 00000000 00:27 61162575 /mnt/huge/rtemap_39 3effe0000000-3effe1000000 rw-s 00000000 00:27 61162576 /mnt/huge/rtemap_40 3effe1000000-3effe2000000 rw-s 00000000 00:27 61162584 /mnt/huge/rtemap_48 3effe2000000-3effe3000000 rw-s 00000000 00:27 61162585 /mnt/huge/rtemap_49 3effe3000000-3effe4000000 rw-s 00000000 00:27 61162586 /mnt/huge/rtemap_50 3effe4000000-3effe5000000 rw-s 00000000 00:27 61162587 /mnt/huge/rtemap_51 3effe5000000-3effe6000000 rw-s 00000000 00:27 61162599 /mnt/huge/rtemap_63 3effe6000000-3effe7000000 rw-s 00000000 00:27 61162598 /mnt/huge/rtemap_62 3effe7000000-3effe8000000 rw-s 00000000 00:27 61162597 /mnt/huge/rtemap_61 3effe8000000-3effe9000000 rw-s 00000000 00:27 61162596 /mnt/huge/rtemap_60 3effe9000000-3effea000000 rw-s 00000000 00:27 61162595 /mnt/huge/rtemap_59 3effea000000-3effeb000000 rw-s 00000000 00:27 61162594 /mnt/huge/rtemap_58 3effeb000000-3effec000000 rw-s 00000000 00:27 61162593 /mnt/huge/rtemap_57 3effec000000-3effed000000 rw-s 00000000 00:27 61162592 /mnt/huge/rtemap_56 3effed000000-3effee000000 rw-s 00000000 00:27 61162591 /mnt/huge/rtemap_55 3effee000000-3effef000000 rw-s 00000000 00:27 61162590 /mnt/huge/rtemap_54 3effef000000-3efff0000000 rw-s 00000000 00:27 61162589 /mnt/huge/rtemap_53 3efff0000000-3efff1000000 rw-s 00000000 00:27 61162588 /mnt/huge/rtemap_52 3efff1000000-3efff2000000 rw-s 00000000 00:27 61162565 /mnt/huge/rtemap_29 3efff2000000-3efff3000000 rw-s 00000000 00:27 61162564 /mnt/huge/rtemap_28 3efff3000000-3efff4000000 rw-s 00000000 00:27 61162563 /mnt/huge/rtemap_27 3efff4000000-3efff5000000 rw-s 00000000 00:27 61162562 /mnt/huge/rtemap_26 3efff5000000-3efff6000000 rw-s 00000000 00:27 61162561 /mnt/huge/rtemap_25 3efff6000000-3efff7000000 rw-s 00000000 00:27 61162560 /mnt/huge/rtemap_24 3efff7000000-3efff8000000 rw-s 00000000 00:27 61162559 /mnt/huge/rtemap_23 3efff8000000-3efff9000000 rw-s 00000000 00:27 61162558 /mnt/huge/rtemap_22 3efff9000000-3efffa000000 rw-s 00000000 00:27 61162557 /mnt/huge/rtemap_21 3efffa000000-3efffb000000 rw-s 00000000 00:27 61162556 /mnt/huge/rtemap_20 3efffb000000-3efffc000000 rw-s 00000000 00:27 61162555 /mnt/huge/rtemap_19 3efffc000000-3efffd000000 rw-s 00000000 00:27 61162554 /mnt/huge/rtemap_18 3efffd000000-3efffe000000 rw-s 00000000 00:27 61162553 /mnt/huge/rtemap_17 3efffe000000-3effff000000 rw-s 00000000 00:27 61162552 /mnt/huge/rtemap_16 3effff000000-3f0000000000 rw-s 00000000 00:27 61162551 /mnt/huge/rtemap_15 3fffb7bc0000-3fffb7c10000 rw-p 00000000 00:00 0=20 3fffb7c10000-3fffb7c50000 rw-s 00000000 00:12 3926240 /run/.rte_config 3fffb7c50000-3fffb7c70000 rw-p 00000000 00:00 0=20 3fffb7c70000-3fffb7e20000 r-xp 00000000 08:32 7090531 /opt/at7.1/lib64/power8/libc-2.19.so 3fffb7e20000-3fffb7e30000 rw-p 001a0000 08:32 7090531 /opt/at7.1/lib64/power8/libc-2.19.so 3fffb7e30000-3fffb7e50000 rw-p 00000000 00:00 0=20 3fffb7e50000-3fffb7e70000 r-xp 00000000 08:32 7090563 /opt/at7.1/lib64/power8/libpthread-2.19.so 3fffb7e70000-3fffb7e80000 rw-p 00010000 08:32 7090563 /opt/at7.1/lib64/power8/libpthread-2.19.so 3fffb7e80000-3fffb7e90000 r-xp 00000000 08:32 7090210 /opt/at7.1/lib64/libdl-2.19.so 3fffb7e90000-3fffb7ea0000 rw-p 00000000 08:32 7090210 /opt/at7.1/lib64/libdl-2.19.so 3fffb7ea0000-3fffb7ec0000 r-xp 00000000 08:32 7090533 /opt/at7.1/lib64/power8/libz.so.1.2.6 3fffb7ec0000-3fffb7ed0000 rw-p 00010000 08:32 7090533 /opt/at7.1/lib64/power8/libz.so.1.2.6 3fffb7ed0000-3fffb7f90000 r-xp 00000000 08:32 7090568 /opt/at7.1/lib64/power8/libm-2.19.so 3fffb7f90000-3fffb7fa0000 rw-p 000b0000 08:32 7090568 /opt/at7.1/lib64/power8/libm-2.19.so 3fffb7fa0000-3fffb7fc0000 r-xp 00000000 00:00 0 [vdso] 3fffb7fc0000-3fffb7ff0000 r-xp 00000000 08:32 7090048 /opt/at7.1/lib64/ld-2.19.so 3fffb7ff0000-3fffb8000000 rw-p 00020000 08:32 7090048 /opt/at7.1/lib64/ld-2.19.so 3ffffffd0000-400000000000 rw-p 00000000 00:00 0 [stack] -----Original Message----- From: Bruce Richardson [mailto:bruce.richardson@intel.com]=20 Sent: 2016=C4=EA3=D4=C223=C8=D5 1:11 To: Sergio Gonzalez Monroy Cc: Gowrishankar ; dev@dpdk.org; chaozhu@linux.vnet.ibm.com; David Marchand Subject: Re: [dpdk-dev] [PATCH] eal/ppc: fix secondary process to map hugepages in correct order On Tue, Mar 22, 2016 at 04:35:32PM +0000, Sergio Gonzalez Monroy wrote: > First of all, forgive my ignorance regarding ppc64 and if the=20 > questions are naive but after having a look to the already existing=20 > code for ppc64 and this patch now, why are we doing this reverse=20 > mapping at all? >=20 > I guess the question revolves around the comment in eal_memory.c: > 1316 /* On PPC64 architecture, the mmap always start = from > higher > 1317 * virtual address to lower address. Here, both = the > physical > 1318 * address and virtual address are in descending order > */ >=20 > From looking at the code, for ppc64 we do qsort in reverse order and=20 > thereafter everything looks to be is done to account for that reverse=20 > sorting. >=20 > CC: Chao Zhu and David Marchand as original author and reviewer of the code. >=20 > Sergio > Just to add my 2c here. At one point, with I believe some i686 installs = - don't remember the specific OS/kernel, we found that the mmap calls were returning the highest free address first and then working downwards - = must like seems to be described here. To fix this we changed the mmap code = from assuming that addresses are mapped upwards, to instead explicitly = requesting a large free block of memory (mmap of /dev/zero) to find a free address space range of the correct size, and then explicitly mmapping each individual page to the appropriate place in that free range. With this scheme it didn't matter whether the OS tried to mmap the pages from the highest or lowest address because we always told the OS where to put the page (and we knew the slot was free from the earlier block mmap). Would this scheme not also work for PPC in a similar way? (Again, = forgive unfamiliarity with PPC! :-) ) /Bruce >=20 > On 07/03/2016 14:13, Gowrishankar wrote: > >From: Gowri Shankar > > > >For a secondary process address space to map hugepages from every=20 > >segment of primary process, hugepage_file entries has to be mapped=20 > >reversely from the list that primary process updated for every=20 > >segment. This is for a reason that, in ppc64, hugepages are sorted = for decrementing addresses. > > > >Signed-off-by: Gowrishankar > >--- > > lib/librte_eal/linuxapp/eal/eal_memory.c | 26 ++++++++++++++++---------- > > 1 file changed, 16 insertions(+), 10 deletions(-) > > > >diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c=20 > >b/lib/librte_eal/linuxapp/eal/eal_memory.c > >index 5b9132c..6aea5d0 100644 > >--- a/lib/librte_eal/linuxapp/eal/eal_memory.c > >+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c > >@@ -1400,7 +1400,7 @@ rte_eal_hugepage_attach(void) > > { > > const struct rte_mem_config *mcfg =3D rte_eal_get_configuration()->mem_config; > > const struct hugepage_file *hp =3D NULL; > >- unsigned num_hp =3D 0; > >+ unsigned num_hp =3D 0, mapped_hp =3D 0; > > unsigned i, s =3D 0; /* s used to track the segment number */ > > off_t size; > > int fd, fd_zero =3D -1, fd_hugepage =3D -1; @@ -1486,14 +1486,12 = @@=20 > >rte_eal_hugepage_attach(void) > > goto error; > > } > >- num_hp =3D size / sizeof(struct hugepage_file); > >- RTE_LOG(DEBUG, EAL, "Analysing %u files\n", num_hp); > >- > > s =3D 0; > > while (s < RTE_MAX_MEMSEG && mcfg->memseg[s].len > 0){ > > void *addr, *base_addr; > > uintptr_t offset =3D 0; > > size_t mapping_size; > >+ unsigned int index; > > #ifdef RTE_LIBRTE_IVSHMEM > > /* > > * if segment has ioremap address set, it's an IVSHMEM segment=20 > >and @@ -1504,6 +1502,8 @@ rte_eal_hugepage_attach(void) > > continue; > > } > > #endif > >+ num_hp =3D mcfg->memseg[s].len / mcfg->memseg[s].hugepage_sz; > >+ RTE_LOG(DEBUG, EAL, "Analysing %u files in segment %u\n", num_hp,=20 > >+s); > > /* > > * free previously mapped memory so we can map the > > * hugepages into the space > >@@ -1514,18 +1514,23 @@ rte_eal_hugepage_attach(void) > > /* find the hugepages for this segment and map them > > * we don't need to worry about order, as the server sorted the > > * entries before it did the second mmap of them */ > >+#ifdef RTE_ARCH_PPC_64 > >+ for (i =3D num_hp-1; i < num_hp && offset < mcfg->memseg[s].len;=20 > >+i--){ #else > > for (i =3D 0; i < num_hp && offset < mcfg->memseg[s].len; i++){ > >- if (hp[i].memseg_id =3D=3D (int)s){ > >- fd =3D open(hp[i].filepath, O_RDWR); > >+#endif > >+ index =3D i + mapped_hp; > >+ if (hp[index].memseg_id =3D=3D (int)s){ > >+ fd =3D open(hp[index].filepath, O_RDWR); > > if (fd < 0) { > > RTE_LOG(ERR, EAL, "Could not open %s\n", > >- hp[i].filepath); > >+ hp[index].filepath); > > goto error; > > } > > #ifdef RTE_EAL_SINGLE_FILE_SEGMENTS > >- mapping_size =3D hp[i].size * hp[i].repeated; > >+ mapping_size =3D hp[index].size * hp[index].repeated; > > #else > >- mapping_size =3D hp[i].size; > >+ mapping_size =3D hp[index].size; > > #endif > > addr =3D mmap(RTE_PTR_ADD(base_addr, offset), > > mapping_size, PROT_READ | PROT_WRITE, @@ -1534,7 +1539,7 @@=20 > >rte_eal_hugepage_attach(void) > > if (addr =3D=3D MAP_FAILED || > > addr !=3D RTE_PTR_ADD(base_addr, offset)) { > > RTE_LOG(ERR, EAL, "Could not mmap %s\n", > >- hp[i].filepath); > >+ hp[index].filepath); > > goto error; > > } > > offset+=3Dmapping_size; > >@@ -1543,6 +1548,7 @@ rte_eal_hugepage_attach(void) > > RTE_LOG(DEBUG, EAL, "Mapped segment %u of size 0x%llx\n", s, > > (unsigned long long)mcfg->memseg[s].len); > > s++; > >+ mapped_hp +=3D num_hp; > > } > > /* unmap the hugepage config file, since we are done using it */ > > munmap((void *)(uintptr_t)hp, size); >=20