From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 56768A05A0; Tue, 21 Apr 2020 22:30:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E0C91C43E; Tue, 21 Apr 2020 22:30:05 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70051.outbound.protection.outlook.com [40.107.7.51]) by dpdk.org (Postfix) with ESMTP id 01AA81C2EB for ; Tue, 21 Apr 2020 22:30:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1iTjhgGoJexxjPUT6HO1PjHC5le6PwhJiGrcLYOLA3M=; b=Fl8VKA6GilwfDYbE8wG+kPuEnTB9Y0yvRCMJVY9lcZJx3a+CydJENh2YayYQi345mK78xeCEtPsyKD1wL0Hin5F6UQf51H1TUQg5w1bZTD3NsdIyFp2YXKSefKZsfbBpq3KPuLNB4UCh1/pm8A40ze71gdGQOA0ayh+2VSAeS1A= Received: from DB6PR0501CA0036.eurprd05.prod.outlook.com (2603:10a6:4:67::22) by DB6PR0802MB2405.eurprd08.prod.outlook.com (2603:10a6:4:9f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.27; Tue, 21 Apr 2020 20:30:00 +0000 Received: from DB5EUR03FT010.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:67:cafe::e5) by DB6PR0501CA0036.outlook.office365.com (2603:10a6:4:67::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.13 via Frontend Transport; Tue, 21 Apr 2020 20:30:00 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT010.mail.protection.outlook.com (10.152.20.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.18 via Frontend Transport; Tue, 21 Apr 2020 20:30:00 +0000 Received: ("Tessian outbound 43fc5cd677c4:v53"); Tue, 21 Apr 2020 20:30:00 +0000 X-CR-MTA-TID: 64aa7808 Received: from e74cd0917f45.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 07DF6275-20BB-4B4F-B826-B29FF43C1FB3.1; Tue, 21 Apr 2020 20:29:55 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e74cd0917f45.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 21 Apr 2020 20:29:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U1WnKDKj+nRgQMjfvbAdJMzVtUxtmwdCd2IYCyduq45p2u7SWdxz4ozIHkR8A3srWGoDxlYF9RSvzuDc2eQ4ZBKf+V9tP80dboeI19Ei478MJ9MqAnvgfwAbXsklO0rvOKZSkqtg5/wBhpfGqiVihgJONFdqLDk/6Zlbp0Bmam9dq9HwhoMBBInpkDD+uCliqRAGFX4gtTpCLpj0uYaF5PvtAaiqLDeqLDaipxVN33h/1PapwEAcATVgiBfWzZqFcStmGlJiAUIp7zg2UYH/j3YeHXsYltQgX2G0zGL8jp9PViNSx7haHe/G6nA+U5IfCOElrO9Fl3y6FBeX/WbI4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1iTjhgGoJexxjPUT6HO1PjHC5le6PwhJiGrcLYOLA3M=; b=PZS4CuBEK842FQfOAVQAMvIyU154n813U8Xok21rA1tvOtWBWBAF1sW85WkcCdA4h03GcsNkejKLYBx+93MKstuL72dtLlSoXBHN+akC5GOtwL10fMpI1NU2ukGPyBmHnXS1bSQHkOJEuHTIiHNj75UOZVSg1ynrTt+sDThvn8hj4bvngsiUZC8+bUBIlqB8xAhH56rksSzcPsnX2noI0wNieIWuSssAagvGPi5NEHz9bIXpRbdIyUNfajrG6Tw8IqTZeEVrqTobmyLxQBBgPBud3Nb2YPcCqSJ1xxs/jps+8wYvoLB2XsLQgFQl5XXRdKh1Kj+WXWVdOL1RvBH8pw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1iTjhgGoJexxjPUT6HO1PjHC5le6PwhJiGrcLYOLA3M=; b=Fl8VKA6GilwfDYbE8wG+kPuEnTB9Y0yvRCMJVY9lcZJx3a+CydJENh2YayYQi345mK78xeCEtPsyKD1wL0Hin5F6UQf51H1TUQg5w1bZTD3NsdIyFp2YXKSefKZsfbBpq3KPuLNB4UCh1/pm8A40ze71gdGQOA0ayh+2VSAeS1A= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) by DBBPR08MB4251.eurprd08.prod.outlook.com (2603:10a6:10:d1::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.29; Tue, 21 Apr 2020 20:29:53 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2921.030; Tue, 21 Apr 2020 20:29:53 +0000 From: Honnappa Nagarahalli To: Aaron Conole , "Ananyev, Konstantin" CC: "dev@dpdk.org" , Gavin Hu , Olivier Matz , nd , Honnappa Nagarahalli , nd Thread-Topic: [dpdk-dev] ISSUE: compiling with asan+ubsan Thread-Index: AQHWF/zUBTTY2Ffg3EaQD0s1FMPWRqiEBYYA Date: Tue, 21 Apr 2020 20:29:53 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 5cafa164-aad4-42b8-810f-ca9f9897be10.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 74f24380-5233-42f2-a9fe-08d7e632c3e3 x-ms-traffictypediagnostic: DBBPR08MB4251:|DBBPR08MB4251:|DB6PR0802MB2405: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 038002787A X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(346002)(136003)(396003)(376002)(39860400002)(366004)(8936002)(33656002)(5660300002)(86362001)(4326008)(186003)(2906002)(71200400001)(8676002)(66476007)(66556008)(7696005)(66446008)(64756008)(66946007)(316002)(76116006)(81156014)(478600001)(54906003)(26005)(110136005)(52536014)(9686003)(55016002)(6506007); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: wixq8UOoR1y5xauTBbWFmS6Y6kawHvFO4chM0Li6KIqk9o/iOPfUgU+LU84KPALAyI3S6iNzGNyq6EmxnnM2PlkQS2CwcspjG3UB6mFT8TNEIhs5egOmVar6T5QM2j9Lwjd8ey3MGMEogH7Y36XndyXHmOcOdQPxmJ8rWkYPpPrrNod1vC/jS0sM5BoG/sHTPW794+IHecXnDbXJ5ua3CZpo2PtVFR/e6iRLmu92gaz8Kcb+3AyIml1yedSQLu+bKLNpeUQd4wY1h3y2gs4n28x7QPUpYHjJNVAyW/UM2ras0fA22F9ZMhLuZydd+RfdBFJWTLFAvRhykT/u0aG2sdjhShpJUj+fMoMt9LVFkEhFrcms2nb5hnBdyDe8bP2ujmIRiBAn2txDIdLP743Ez6N0l0PUGYt89eov8HA04xOWyFj4sWGdok86LErruJ9a x-ms-exchange-antispam-messagedata: gExdrXiv/EIcNIc6Jp9QGuIRCLLbn8q4LRyx10BRkA8wOXlDij3pXiUiWkNR3yUefRGbAJLuixWhyGJNnmzP0cGB5bx+bbrBEMb59AYQWR6O0rqeMFyBr/evN+yTrhMpdpeRPeMHcI0u89Tzaohgqw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4251 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT010.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(396003)(39860400002)(346002)(376002)(136003)(46966005)(70206006)(52536014)(9686003)(6506007)(8936002)(336012)(316002)(4326008)(86362001)(5660300002)(82310400002)(70586007)(55016002)(81156014)(8676002)(81166007)(33656002)(478600001)(356005)(7696005)(110136005)(82740400003)(47076004)(2906002)(54906003)(186003)(26005); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: ac2f37c6-d336-4650-5385-08d7e632bfb2 X-Forefront-PRVS: 038002787A X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h+N2ihGMNuNeeMoDXO9uINuABPBdrPMuo3KalJ2SWk0x8lcUpfwI1ssxpXcv+GRhhAwHtgVYW3f/voXWkktyC+1BMxZcBqLoWkIp2+P1P6WN2Mw18MI27lnp+WI6rOj0aHc3/YIVXBku+hjX6F91sJ9B+6juXQbxHhAzgg52IfIMl9fHCazvFXAS65dQ1DbKeQgHL5wI99Z9d0eNM/RR3FvbgNKT8Cz16Uoy7dBwdy8I1ADAMPL172862gJHYEegNxtNdc40PKoo0b8PoOTOPGVKGxajPUNKMMQ0zxLd/R5g1LHeK7QZAx3gkZkpqNoX21MqTCeZCuR9MqLJVE8J9m2Z/j3FlDi5PAU2XZ67LgIYdfixSQUbh4Ykzd3Er/927bPCDcvQx3gOZbSyP2Se06UmnfbVi3yyy3xs/VQ0UnQK4EwON5p82Ghy8d1Rg1q499ZAZYcuGyWRrv6vJaX0qPFlQALFAHHDmwROtavTbJ8bqb3ijfYb0S5HpABH3iITbmidh8iufeMU3r3xfDq49Q== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2020 20:30:00.5375 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 74f24380-5233-42f2-a9fe-08d7e632c3e3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2405 Subject: Re: [dpdk-dev] ISSUE: compiling with asan+ubsan X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > "Ananyev, Konstantin" writes: >=20 > > Hi Aaron, > > > >> While compiling with asan and ubsan I run into the following error: > >> > >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o > >> > >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test > >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config > >> - > >> Ilib/librte_eal/include -I../lib/librte_eal/include > >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include > >> -Ilib/librte_eal/common - I../lib/librte_eal/common > >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include > >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - > >> I../lib/librte_kvargs -Ilib/librte_bitratestats > >> -I../lib/librte_bitratestats -Ilib/librte_ethdev > >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net > >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool > >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - > >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics > >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf > >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline > >> -I../lib/librte_cmdline -Ilib/librte_cryptodev > >> -I../lib/librte_cryptodev -Ilib/librte_distributor - > >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd > >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev > >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer > >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib > >> -I../lib/librte_rib -Ilib/librte_flow_classify - > >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table > >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched > >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag > >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci > >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - > >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security > >> -I../lib/librte_security -Ilib/librte_latencystats > >> -I../lib/librte_latencystats - Ilib/librte_member > >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline > >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - > >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder > >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - > >> I../drivers/mempool/ring -Idrivers/mempool/stack > >> -I../drivers/mempool/stack -Idrivers/event/skeleton > >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci > >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev > >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring > >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power > >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev > >> -I../lib/librte_compressdev -fdiagnostics-color=3Dalways > >> -fsanitize=3Daddress,undefined -fno-omit-frame-pointer -pipe > >> -D_FILE_OFFSET_BITS=3D64 -Wall -Winvalid-pch -Werror -O2 -g -include > >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral > >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes > >> -Wnested-externs -Wold-style-definition -Wpointer-arith - > >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings > >> -Wno-missing-field-initializers -march=3Dnative -mno-avx512f - > >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - > MD -MQ > >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' > >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o > >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c > >> ../app/test/test_ring.c > >> > >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 > >> bytes I have a machine where I hit this error for a lot of other files every time= I do a fresh compile. I attribute that to low swap space. > > > > I also noticed that test_ring.c compilation takes a huge amount of time= and > memory. > > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but stil= l The compilation time issue was attributed to compiler version. We saw the i= ssues only on clang. GCC did not show any problems. Is this problem something new? > seems too much. > > Will try to have a look later this week. >=20 > Okay - glad I'm not the only one. My only theory is that all the inlinin= g is > exploding the size of the file during the translation unit processing. >=20 > I ran it through the preprocessor and didn't see any large arrays created > anywhere. But there are loads of calls to the rte_ring that are tagged a= s > "always inline" - even the test_enqueue and test_dequeue functions are > "always inline" and they are quite large. Just a completely unfounded gu= ess. >=20 > CC'd Honnappa (and others) to take a look. >=20 > >> > >> This is in a constrained (read: container) environment. I guess one > >> way of resolving would be to allocate more memory to the container, > >> but I'm also curious why the object files are getting so large? > >> Should I consider this a bug or "working as intended"? This will > >> have implications if we want asan/ubsan under the travis build also. > >> > >> -Aaron