From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id DF9E2A0C4B;
	Sat,  6 Nov 2021 14:11:33 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 5AF6440151;
	Sat,  6 Nov 2021 14:11:33 +0100 (CET)
Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com
 [66.111.4.29]) by mails.dpdk.org (Postfix) with ESMTP id BF35940040
 for <dev@dpdk.org>; Sat,  6 Nov 2021 14:11:31 +0100 (CET)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailout.nyi.internal (Postfix) with ESMTP id 1E5F15C0125;
 Sat,  6 Nov 2021 09:11:31 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Sat, 06 Nov 2021 09:11:31 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h=
 from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding:content-type; s=fm2; bh=
 7kBxO9Rp+8IlZxrHP4747zs/2uiTuQT9h/ZryjwJ3G0=; b=o2ycgg0RSNGPN7HN
 Py0ZpLZktgxwbazeZZ/1MC4qJ7fPJxl8UGKOI+nudVOZ3noTAgxJfURo17BUq1Pf
 Z6L3f0gtnYjoOPduZR5mpq45whjzogAt2FLL08w5wshdnT1iDTuHiEF2ct87sts0
 71u+MVSe6DaywAZgDx6EIPGhT0vURRbvuzYA4UGeVpKTsfdLe4PnF4RDRUGeN8cc
 kdx6yNXi/Kx4YblLcoOX25JNiIXi9MS+Tlkzk+0tzYCMe9W+/4CiLyXxkrRkFlSa
 wVbMtUzsl7AHcyz3lJxs4E8m+pasQqL/pGEw1N7FUhZTz54PuBRcvify4leNeR0W
 ua/qaA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-transfer-encoding:content-type
 :date:from:in-reply-to:message-id:mime-version:references
 :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
 :x-sasl-enc; s=fm1; bh=7kBxO9Rp+8IlZxrHP4747zs/2uiTuQT9h/ZryjwJ3
 G0=; b=KFd3JwyCebojVqWWs4K3BUq9s+g5TEaY9/vs/+VqL3T/5Ry17XSrwVp87
 J5zj9Fq67p3fqWu8lCSEyCb5+YocVS0GwjWGz9lB+KpI8k1E00LSYNeyhyZiLX2Y
 Qb70N0o2Je0c/J1C6G4ZUJWFhBi1K/84l0iNpd64uKCGJmhIAuKYi36ZQbCOzQD/
 LcGIt7SxfDe9vvuu9t1KdhBDW4mkZ82V1mv0vcNajP/VLjNoLTdH2w0yl/4OcOPd
 wPPBEpbkdugeEPsruX9mntcNAcuep2IJCHdpPwcIt0RvqiGKSPrH73djJEChA+em
 fbATppKsABxeJpxCUoAZHaWONN7Yw==
X-ME-Sender: <xms:AX-GYZM3M6kz-wb7_V7eRsCTP_X5GnDOImmYcMLPi1lu74QLJ3pilQ>
 <xme:AX-GYb_XNgxjKoZ83z4W55BRexEj29cB1muzd-V9HvNRJZRlCADoJwenCnD8IRrri
 0piXvXJqVeukPdFLw>
X-ME-Received: <xmr:AX-GYYTOo_ZJq7k7qBuGcrOQ2uY-tHcn6zsyLxbfXWSApBY1raogu93UASDCqu9lxSxp7AwRfuSwwUzQPGMqxtmNDw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrtdekgdegjecutefuodetggdotefrodftvf
 curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
 uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
 fjughrpefhvffufffkjghfggfgtgesthhqredttddtjeenucfhrhhomhepvfhhohhmrghs
 ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf
 frrghtthgvrhhnpeekteehtdeivefhieegjeelgedufeejheekkeetueevieeuvdevuedt
 jeevheevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh
 hmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght
X-ME-Proxy: <xmx:AX-GYVvy3E2T1nySwoAB7aR5Tt6DXFSo3FEboRoSQQr3Km05KsMtNA>
 <xmx:AX-GYRe3-DqJjoJRveObb0nfaZqpN-YQRQAlKmWg15apFLqX0xKIPg>
 <xmx:AX-GYR07f8ztVHoK3e19w4bfFaH5wRkVXgFK435jOO_9aCf7g74EaQ>
 <xmx:A3-GYSrw5I8DsSRZ3tACn51OlX0nFW0LdeY_kiyNGi53X6XBKC_yNg>
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
 6 Nov 2021 09:11:29 -0400 (EDT)
From: Thomas Monjalon <thomas@monjalon.net>
To: "Min Hu (Connor)" <humin29@huawei.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Ferruh Yigit <ferruh.yigit@intel.com>,
 jpf@zurich.ibm.com
Date: Sat, 06 Nov 2021 14:11:27 +0100
Message-ID: <1669975.jV5p3Ss03J@thomas>
In-Reply-To: <58e86979-cc9b-7dce-edf7-65867a8d4f8d@huawei.com>
References: <58e86979-cc9b-7dce-edf7-65867a8d4f8d@huawei.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="UTF-8"
Subject: Re: [dpdk-dev] Questions about mmap in Asan APP
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

06/11/2021 10:27, Min Hu (Connor):
> Hi, Jonas Pfefferle and all,
> 	I met one questions:
> 	When I ran APP on Kasan OS, secondary process occasionally init
> failed.
>=20
> BTW:
> testpmd is built with Asan, that is to add "CFLAGS=3D"-fsanitize=3Daddres=
s=20
> -fno-omit-frame-pointer -pthread"".
> Kasan OS is what opens this:
> CONFIG_SLUB_DEBUG=3Dy
> CONFIG_KASAN=3Dy
>=20
> The PRIMARY process APP(testpmd) log:
> estuary:/home$ ./testpmd_kasan_debug    -w 0000:00:01.0=20
> --file-prefix=3Dtest --proc-type=3Dauto -l 0-1 -n 2 -- -i --rxq=3D4 --txq=
=3D4
> EAL: Detected 16 lcore(s)
> EAL: Detected 1 NUMA nodes
> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
> EAL: Auto-detected process type: PRIMARY
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/test/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available 32768 kB hugepages reported
> EAL: No available 64 kB hugepages reported
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
>=20
> EAL: WARNING! Base virtual address hint (0xd01061000 !=3D 0xfffb89e00000)=
=20
> not respected!
> EAL:    This may cause issues with mapping memory into secondary processes
>=20
> EAL:   Invalid NUMA socket, default to 0
> EAL: Using IOMMU type 8 (No-IOMMU)
> EAL: Probe PCI driver: net_hns3_vf (19e5:a22e) device: 0000:00:01.0=20
> (socket 0)
> TELEMETRY: No legacy callbacks, legacy socket not created
> Interactive-mode selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=3D155456, size=3D2176, soc=
ket=3D0
> testpmd: preferred mempool ops selected: ring_mp_mc
>=20
> Warning! port-topology=3Dpaired and odd forward ports number, the last=20
> port will pair with itself.
>=20
> Configuring Port 0 (socket 0)
> 0000:00:01.0 hns3_parse_rss_key(): Default RSS hash key to be set
>=20
> Port 0: link state change event
> Port 0: 72:DE:F7:1B:F6:52
> Checking link statuses...
>=20
> "EAL: WARNING! Base virtual address hint (0xd01061000 !=3D 0xfffb89e00000=
)=20
> not respected!  EAL:    This may cause issues with mapping memory into
> secondary processes" should be noticed.
>=20
>=20
>=20
> The SECONDARY APP(proc-info) log is:
> estuary:/home$ ./proc-info -w 0000:00:01.0 --file-prefix=3Dtest=20
> --proc-type=3Dauto -- --xstats-name rx_q0_packets
> Option -w, --pci-whitelist is deprecated, use -a, --allow option instead
> EAL: Cannot get a virtual area at requested address: 0xfffba6000000 (got=
=20
> 0xfffb8ae3b000)
>=20
> EAL: Cannot reserve 17179869184 bytes at [0xfffba6000000] - please use=20
> '--base-virtaddr' option
>=20
> EAL: Cannot preallocate VA space for hugepage memory
> EAL: FATAL: Cannot init memory
> EAL: Cannot init memory
> PANIC in main():
> Cannot init EAL
> 4: [/lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe0)=20
> [0xffff8be5a8a0]]
> 3: [./proc-info(main+0x1098) [0x52a3a8]]
> 2: [./proc-info(__rte_panic+0x9c) [0x48cfac]]
> 1: [./proc-info(rte_dump_stack+0x20) [0x80d4b0]]
> Aborted
>=20
> "EAL: Cannot get a virtual area at requested address: 0xfffba6000000=20
> (got 0xfffb8ae3b000)" should be noticed.
>=20
> According to my analysis, the reason is that the system call "mmap"
> return the address which is different with the hint address(first
> parameter). That is to say=EF=BC=9A
> return_addr =3D mmap(void* start,size_t length,int prot,int flags,int
> fd,off_t offset);
> return_addr and start is different.
>=20
> Why this happened? I guess Asan works based on shadow memory, shadow
> memory will occupy the memory zone and it cannot be one for mmap.
> So, it the shadow memory happens to be in the range of mmap region,
> mmap will return a new address which is free(not occupied by shadow).
>=20
> BTW=EF=BC=8CI think DPDK multiple process is not supported in that situat=
ion,
> such as "Asan APP on Kasan OS".
>=20
> What is your opinion? any will weclome, thanks.

Secondary process mapping is fragile.
I am not surprised there are more issues when enabling
some kernel features playing with memory ranges.
You may try to find more hints to make it work,
but it is OK stating this scenario is not supported.

Another outcome to this trial could be to document the limitation
in DPDK documentation for Linux.

Thanks