From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f196.google.com (mail-wr0-f196.google.com [209.85.128.196]) by dpdk.org (Postfix) with ESMTP id 998AA2BB9 for ; Fri, 9 Jun 2017 14:08:29 +0200 (CEST) Received: by mail-wr0-f196.google.com with SMTP id v104so6380199wrb.0 for ; Fri, 09 Jun 2017 05:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=WtvYHZpPdbVEuIYBSIautOolo0fAZioEpWG0ye9llQw=; b=HpggxnNUPkdVE1QB7aArKcZmTcmCr/Zl1LqGFSEvPivGFGaBtkjS//uDye9iz9kuiO CdrDKFbNZB6v3Uiesg5hP7QasHM5jf8gcxuX1QXBiPHWXB+ly4TfcqjmvaMRJhRugVXD FKqyfX8N5KicmOT/ZsPmzzzDTFwLaJ2QOEdlr1R+Nuw9x/v/hbszyml2ACP5dASlU54a PbS5+wtdvAJkETjfvTLyNJ/QNM7DHmPST4tqPjPXbtmSGH/JgpJHysmQJcOzlWpt75dc MS5m+hsU7/oPGKvXzsxsYHyXVeNBCZwpnjbamhudjW30nmCsZaFbBbiC25x6VRKojFFU 1hpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=WtvYHZpPdbVEuIYBSIautOolo0fAZioEpWG0ye9llQw=; b=EmkMcRO7kQigLhoQ7ekZNW4KCXnfnMFWgU8hV96qX+pHXVvtvvDtEXlc07Sh9RX8+l 20Q9f8txoPcqqCDOm6kymnkeZ4FkntFfxOeyHm3PDnZurTWDHdanKbKkTUpiv/1QiORh bgVDH6lnR9P01HAoiuR5raqa0+rxVlgZJ6deKusGbpHvrpY4KynkPMYqos6ZmMu21t7g 1btEjb8YOHwvQD7rRolMgQtjMpPxDQlqVQStPpDRUJzLA7CetfmaBND+C5t/dwtyqlY+ VXNZdYt+anKzCwcQxbpwQv4U95KdWj0+otIEjS3RsKljZ0KnD3v1la2UhDV/mwqh126d N8vA== X-Gm-Message-State: AODbwcBta6PnkXWb5qTPcgQNxmj+Osir+3NABwJ6d/jg41oxONXgOvlc 3eelOCcAW66f9h2GyAo= X-Received: by 10.28.227.67 with SMTP id a64mr6929544wmh.43.1497010109075; Fri, 09 Jun 2017 05:08:29 -0700 (PDT) Received: from [10.1.0.103] (bba193486.alshamil.net.ae. [217.165.96.192]) by smtp.gmail.com with ESMTPSA id n2sm1591607wmd.16.2017.06.09.05.08.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Jun 2017 05:08:28 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Ilya Matveychikov In-Reply-To: <20170609102727.0eb7f39d@platinum> Date: Fri, 9 Jun 2017 16:08:24 +0400 Cc: dev@dpdk.org, "adrien.mazarguil@6wind.com" , Jan Blunck , Sergio Gonzalez Monroy Content-Transfer-Encoding: quoted-printable Message-Id: <80B1FDDC-4B0B-4B12-843F-B3AB1587CD08@gmail.com> References: <1A9D36CE-8B6D-43A7-BE0C-9F232DFDA263@gmail.com> <20170609102727.0eb7f39d@platinum> To: Olivier Matz X-Mailer: Apple Mail (2.3273) Subject: Re: [dpdk-dev] A (possible) problem with `--no-huge` option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jun 2017 12:08:29 -0000 Hi Olivier, The patch from you solves the problem for me. Thank you. > On Jun 9, 2017, at 12:27 PM, Olivier Matz = wrote: >=20 > Hi Ilya, >=20 > On Sun, 14 May 2017 14:34:14 +0400, Ilya Matveychikov = wrote: >> Hi guys, >>=20 >> I have a problem while running DPDK with `--no-huge` option. It seems = that the problem occurs since commit = cdc242f260e766bd95a658b5e0686a62ec04f5b0 and that is the change that = affects me: >>=20 >> + if ((page & 0x7fffffffffffffULL) =3D=3D 0) >> + return RTE_BAD_PHYS_ADDR; >> + >>=20 >> What I did is to try to create memory pool using = rte_pktmbuf_pool_create(). I dig into the issue and found that in my = case =E2=80=9Cpage" value is 0x0080000000000000 which means that the = page is not present and =E2=80=9Csoft-dirty=E2=80=9D (according to = kernel=E2=80=99s documentation): >>=20 >> * Bits 0-54 page frame number (PFN) if present >> * Bits 0-4 swap type if swapped >> * Bits 5-54 swap offset if swapped >> * Bit 55 pte is soft-dirty (see = Documentation/vm/soft-dirty.txt) >> * Bit 56 page exclusively mapped (since 4.2) >> * Bits 57-60 zero >> * Bit 61 page is file-page or shared-anon (since 3.5) >> * Bit 62 page swapped >> * Bit 63 page present >>=20 >> So, before the change mentioned all =E2=80=9Cworks=E2=80=9D fine and = such pages were not handled. But now the check causes = rte_mempool_populate_default to fail with -EINVAL... >> Can anyone familiar with the memory pool allocation helps with the = issue? >>=20 >> Thanks in advice, >> Ilya Matveychikov. >>=20 >=20 > I can reproduce the issue: >=20 > make config T=3Dx86_64-native-linuxapp-gcc > make -j32 EXTRA_CFLAGS=3D"-O0 -g" > mkdir -p /mnt/huge > mount -t hugetlbfs nodev /mnt/huge > echo 256 > = /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages >=20 > # ok > ./build/app/testpmd -l 2,4 --log-level 8 --vdev=3Deth_null0 -- = --no-numa --total-num-mbufs=3D4096 -i --port-topology=3Dchained >=20 > # fail > ./build/app/testpmd --no-huge -l 2,4 --log-level 8 --vdev=3Deth_null0 = -- --no-numa --total-num-mbufs=3D4096 -i --port-topology=3Dchained >=20 >=20 > I confirm that rte_mem_virt2phy() returns RTE_BAD_PHYS_ADDR, > which makes rte_mempool_populate_virt() to fail. >=20 > Reverting cdc242f260e7 ("eal/linux: support running as unprivileged = user") > fixes the problem. Actually, it makes rte_mem_virt2phy() return 0 = instead > of RTE_BAD_PHYS_ADDR, which is seen as a valid address. >=20 > I think querying the physical address when using --no-huge does not = make > sense because the memory is not locked, and could be swapped. >=20 > Another strange thing, when using --no-huge, the physical address = returned > when allocating a memzone is the virtual address. >=20 > I see several solutions to fix the issue: >=20 > 1/ Always set physical addresses to RTE_BAD_PHYS_ADDR when started > with --no-huge. We consider that the physical address is invalid > in that case and must not be used. >=20 > This impacts rte_mem_virt2phy() and memzone_reserve*() functions. >=20 > In rte_mempool_populate_virt(), don't expect a physical address > if the application is started with --no-huge. >=20 > 2/ Change rte_mem_virt2phy() to return the virtual address when we > ask for the physical address when started with --no-huge. This is > wrong, but consistent with what is done in memzones today. >=20 > In rte_mem_virt2phy(), add at the beginning: >=20 > if (!rte_eal_has_hugepages()) > return (intptr_t)virtaddr; >=20 > 3/ lock pages in memory by reverting > 729f17a932dd ("mem: revert page locking when not using hugepages") >=20 > This would make the physical address available. > As explained in the commit log, this would also break the ability to > start dpdk with --no-huge for non-root users. >=20 >=20 > I think 1/ is better. I'm sending a patch in reply to this mail. > Ilya, please let me know if it fixes your issue. >=20 > Regards, > Olivier