Hi all, While compiling with asan and ubsan I run into the following error: FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config -Ilib/librte_eal/include -I../lib/librte_eal/include -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include -Ilib/librte_eal/common -I../lib/librte_eal/common -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs -I../lib/librte_kvargs -Ilib/librte_bitratestats -I../lib/librte_bitratestats -Ilib/librte_ethdev -I../lib/librte_ethdev -Ilib/librte_net -I../lib/librte_net -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring -Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf -Ilib/librte_cfgfile -I../lib/librte_cfgfile -Ilib/librte_cmdline -I../lib/librte_cmdline -Ilib/librte_cryptodev -I../lib/librte_cryptodev -Ilib/librte_distributor -I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev -I../lib/librte_eventdev -Ilib/librte_timer -I../lib/librte_timer -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib -I../lib/librte_rib -Ilib/librte_flow_classify -I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched -I../lib/librte_sched -Ilib/librte_ip_frag -I../lib/librte_ip_frag -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm -Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security -I../lib/librte_security -Ilib/librte_latencystats -I../lib/librte_latencystats -Ilib/librte_member -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu -I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring -I../drivers/mempool/ring -Idrivers/mempool/stack -I../drivers/mempool/stack -Idrivers/event/skeleton -I../drivers/event/skeleton -Idrivers/bus/pci -I../drivers/bus/pci -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev -Idrivers/net/bonding -I../drivers/net/bonding -Idrivers/net/ring -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev -I../lib/librte_compressdev -fdiagnostics-color=always -fsanitize=address,undefined -fno-omit-frame-pointer -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-missing-field-initializers -march=native -mno-avx512f -DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE -MD -MQ 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c ../app/test/test_ring.c cc1: out of memory allocating 65536 bytes after a total of 4609626112 bytes This is in a constrained (read: container) environment. I guess one way of resolving would be to allocate more memory to the container, but I'm also curious why the object files are getting so large? Should I consider this a bug or "working as intended"? This will have implications if we want asan/ubsan under the travis build also. -Aaron
Hi Aaron, > While compiling with asan and ubsan I run into the following error: > > FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o > > gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config - > Ilib/librte_eal/include -I../lib/librte_eal/include -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include -Ilib/librte_eal/common - > I../lib/librte_eal/common -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - > I../lib/librte_kvargs -Ilib/librte_bitratestats -I../lib/librte_bitratestats -Ilib/librte_ethdev -I../lib/librte_ethdev -Ilib/librte_net - > I../lib/librte_net -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - > Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf -Ilib/librte_cfgfile - > I../lib/librte_cfgfile -Ilib/librte_cmdline -I../lib/librte_cmdline -Ilib/librte_cryptodev -I../lib/librte_cryptodev -Ilib/librte_distributor - > I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev -I../lib/librte_eventdev - > Ilib/librte_timer -I../lib/librte_timer -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib -I../lib/librte_rib -Ilib/librte_flow_classify - > I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched -I../lib/librte_sched - > Ilib/librte_ip_frag -I../lib/librte_ip_frag -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - > Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security -I../lib/librte_security -Ilib/librte_latencystats -I../lib/librte_latencystats - > Ilib/librte_member -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - > I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - > I../drivers/mempool/ring -Idrivers/mempool/stack -I../drivers/mempool/stack -Idrivers/event/skeleton -I../drivers/event/skeleton - > Idrivers/bus/pci -I../drivers/bus/pci -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev -Idrivers/net/bonding - > I../drivers/net/bonding -Idrivers/net/ring -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power -Ilib/librte_pdump -I../lib/librte_pdump > -Ilib/librte_compressdev -I../lib/librte_compressdev -fdiagnostics-color=always -fsanitize=address,undefined -fno-omit-frame-pointer -pipe > -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- > nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith - > Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-missing-field-initializers -march=native -mno-avx512f - > DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE -MD -MQ 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' > -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c > ../app/test/test_ring.c > > cc1: out of memory allocating 65536 bytes after a total of 4609626112 > bytes I also noticed that test_ring.c compilation takes a huge amount of time and memory. On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still seems too much. Will try to have a look later this week. > > This is in a constrained (read: container) environment. I guess one way > of resolving would be to allocate more memory to the container, but I'm > also curious why the object files are getting so large? Should I > consider this a bug or "working as intended"? This will have > implications if we want asan/ubsan under the travis build also. > > -Aaron
"Ananyev, Konstantin" <konstantin.ananyev@intel.com> writes: > Hi Aaron, > >> While compiling with asan and ubsan I run into the following error: >> >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o >> >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config >> - >> Ilib/librte_eal/include -I../lib/librte_eal/include >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include >> -Ilib/librte_eal/common - >> I../lib/librte_eal/common -Ilib/librte_eal/x86/include >> -I../lib/librte_eal/x86/include -Ilib/librte_eal -I../lib/librte_eal >> -Ilib/librte_kvargs - >> I../lib/librte_kvargs -Ilib/librte_bitratestats >> -I../lib/librte_bitratestats -Ilib/librte_ethdev >> -I../lib/librte_ethdev -Ilib/librte_net - >> I../lib/librte_net -Ilib/librte_mbuf -I../lib/librte_mbuf >> -Ilib/librte_mempool -I../lib/librte_mempool -Ilib/librte_ring >> -I../lib/librte_ring - >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf >> -Ilib/librte_cfgfile - >> I../lib/librte_cfgfile -Ilib/librte_cmdline -I../lib/librte_cmdline >> -Ilib/librte_cryptodev -I../lib/librte_cryptodev >> -Ilib/librte_distributor - >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev >> -I../lib/librte_eventdev - >> Ilib/librte_timer -I../lib/librte_timer -Ilib/librte_fib >> -I../lib/librte_fib -Ilib/librte_rib -I../lib/librte_rib >> -Ilib/librte_flow_classify - >> I../lib/librte_flow_classify -Ilib/librte_table >> -I../lib/librte_table -Ilib/librte_port -I../lib/librte_port >> -Ilib/librte_sched -I../lib/librte_sched - >> Ilib/librte_ip_frag -I../lib/librte_ip_frag -Ilib/librte_kni >> -I../lib/librte_kni -Ilib/librte_pci -I../lib/librte_pci >> -Ilib/librte_lpm -I../lib/librte_lpm - >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security >> -I../lib/librte_security -Ilib/librte_latencystats >> -I../lib/librte_latencystats - >> Ilib/librte_member -I../lib/librte_member -Ilib/librte_pipeline >> -I../lib/librte_pipeline -Ilib/librte_rawdev -I../lib/librte_rawdev >> -Ilib/librte_rcu - >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - >> I../drivers/mempool/ring -Idrivers/mempool/stack >> -I../drivers/mempool/stack -Idrivers/event/skeleton >> -I../drivers/event/skeleton - >> Idrivers/bus/pci -I../drivers/bus/pci -I../drivers/bus/pci/linux >> -Idrivers/bus/vdev -I../drivers/bus/vdev -Idrivers/net/bonding - >> I../drivers/net/bonding -Idrivers/net/ring -I../drivers/net/ring >> -Ilib/librte_power -I../lib/librte_power -Ilib/librte_pdump >> -I../lib/librte_pdump >> -Ilib/librte_compressdev -I../lib/librte_compressdev >> -fdiagnostics-color=always -fsanitize=address,undefined >> -fno-omit-frame-pointer -pipe >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- >> nonliteral -Wformat-security -Wmissing-declarations >> -Wmissing-prototypes -Wnested-externs -Wold-style-definition >> -Wpointer-arith - >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings >> -Wno-missing-field-initializers -march=native -mno-avx512f - >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE -MD -MQ >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c >> ../app/test/test_ring.c >> >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 >> bytes > > I also noticed that test_ring.c compilation takes a huge amount of time and memory. > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still seems too much. > Will try to have a look later this week. Okay - glad I'm not the only one. My only theory is that all the inlining is exploding the size of the file during the translation unit processing. I ran it through the preprocessor and didn't see any large arrays created anywhere. But there are loads of calls to the rte_ring that are tagged as "always inline" - even the test_enqueue and test_dequeue functions are "always inline" and they are quite large. Just a completely unfounded guess. CC'd Honnappa (and others) to take a look. >> >> This is in a constrained (read: container) environment. I guess one way >> of resolving would be to allocate more memory to the container, but I'm >> also curious why the object files are getting so large? Should I >> consider this a bug or "working as intended"? This will have >> implications if we want asan/ubsan under the travis build also. >> >> -Aaron
<snip> > > "Ananyev, Konstantin" <konstantin.ananyev@intel.com> writes: > > > Hi Aaron, > > > >> While compiling with asan and ubsan I run into the following error: > >> > >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o > >> > >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test > >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config > >> - > >> Ilib/librte_eal/include -I../lib/librte_eal/include > >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include > >> -Ilib/librte_eal/common - I../lib/librte_eal/common > >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include > >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - > >> I../lib/librte_kvargs -Ilib/librte_bitratestats > >> -I../lib/librte_bitratestats -Ilib/librte_ethdev > >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net > >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool > >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - > >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics > >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf > >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline > >> -I../lib/librte_cmdline -Ilib/librte_cryptodev > >> -I../lib/librte_cryptodev -Ilib/librte_distributor - > >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd > >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev > >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer > >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib > >> -I../lib/librte_rib -Ilib/librte_flow_classify - > >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table > >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched > >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag > >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci > >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - > >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security > >> -I../lib/librte_security -Ilib/librte_latencystats > >> -I../lib/librte_latencystats - Ilib/librte_member > >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline > >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - > >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder > >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - > >> I../drivers/mempool/ring -Idrivers/mempool/stack > >> -I../drivers/mempool/stack -Idrivers/event/skeleton > >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci > >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev > >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring > >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power > >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev > >> -I../lib/librte_compressdev -fdiagnostics-color=always > >> -fsanitize=address,undefined -fno-omit-frame-pointer -pipe > >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include > >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral > >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes > >> -Wnested-externs -Wold-style-definition -Wpointer-arith - > >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings > >> -Wno-missing-field-initializers -march=native -mno-avx512f - > >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - > MD -MQ > >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' > >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o > >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c > >> ../app/test/test_ring.c > >> > >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 > >> bytes I have a machine where I hit this error for a lot of other files every time I do a fresh compile. I attribute that to low swap space. > > > > I also noticed that test_ring.c compilation takes a huge amount of time and > memory. > > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still The compilation time issue was attributed to compiler version. We saw the issues only on clang. GCC did not show any problems. Is this problem something new? > seems too much. > > Will try to have a look later this week. > > Okay - glad I'm not the only one. My only theory is that all the inlining is > exploding the size of the file during the translation unit processing. > > I ran it through the preprocessor and didn't see any large arrays created > anywhere. But there are loads of calls to the rte_ring that are tagged as > "always inline" - even the test_enqueue and test_dequeue functions are > "always inline" and they are quite large. Just a completely unfounded guess. > > CC'd Honnappa (and others) to take a look. > > >> > >> This is in a constrained (read: container) environment. I guess one > >> way of resolving would be to allocate more memory to the container, > >> but I'm also curious why the object files are getting so large? > >> Should I consider this a bug or "working as intended"? This will > >> have implications if we want asan/ubsan under the travis build also. > >> > >> -Aaron
Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> writes: > <snip> > >> >> "Ananyev, Konstantin" <konstantin.ananyev@intel.com> writes: >> >> > Hi Aaron, >> > >> >> While compiling with asan and ubsan I run into the following error: >> >> >> >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o >> >> >> >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test >> >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config >> >> - >> >> Ilib/librte_eal/include -I../lib/librte_eal/include >> >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include >> >> -Ilib/librte_eal/common - I../lib/librte_eal/common >> >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include >> >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - >> >> I../lib/librte_kvargs -Ilib/librte_bitratestats >> >> -I../lib/librte_bitratestats -Ilib/librte_ethdev >> >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net >> >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool >> >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - >> >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics >> >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf >> >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline >> >> -I../lib/librte_cmdline -Ilib/librte_cryptodev >> >> -I../lib/librte_cryptodev -Ilib/librte_distributor - >> >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd >> >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev >> >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer >> >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib >> >> -I../lib/librte_rib -Ilib/librte_flow_classify - >> >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table >> >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched >> >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag >> >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci >> >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - >> >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security >> >> -I../lib/librte_security -Ilib/librte_latencystats >> >> -I../lib/librte_latencystats - Ilib/librte_member >> >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline >> >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - >> >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder >> >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - >> >> I../drivers/mempool/ring -Idrivers/mempool/stack >> >> -I../drivers/mempool/stack -Idrivers/event/skeleton >> >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci >> >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev >> >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring >> >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power >> >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev >> >> -I../lib/librte_compressdev -fdiagnostics-color=always >> >> -fsanitize=address,undefined -fno-omit-frame-pointer -pipe >> >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include >> >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral >> >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes >> >> -Wnested-externs -Wold-style-definition -Wpointer-arith - >> >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings >> >> -Wno-missing-field-initializers -march=native -mno-avx512f - >> >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - >> MD -MQ >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' >> >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c >> >> ../app/test/test_ring.c >> >> >> >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 >> >> bytes > I have a machine where I hit this error for a lot of other files every > time I do a fresh compile. I attribute that to low swap space. I am shocked that a translation unit should be 4.6G? That's unbelievably HUGE - there's no way that's correct. There must be something going on with your machine - I've only ever seen it with test_ring.c and it's very consistent. >> > >> > I also noticed that test_ring.c compilation takes a huge amount of time and >> memory. >> > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still > The compilation time issue was attributed to compiler version. We saw > the issues only on clang. GCC did not show any problems. > Is this problem something new? This is happening on GCC. Actually, the meson build with clang + asan&ubsan doesn't even work right. >> seems too much. >> > Will try to have a look later this week. >> >> Okay - glad I'm not the only one. My only theory is that all the inlining is >> exploding the size of the file during the translation unit processing. >> >> I ran it through the preprocessor and didn't see any large arrays created >> anywhere. But there are loads of calls to the rte_ring that are tagged as >> "always inline" - even the test_enqueue and test_dequeue functions are >> "always inline" and they are quite large. Just a completely unfounded guess. As a test, I changed the attributes in test_ring.h to remove the always_inline - and I was finally able to complete a build with asan&ubsan using GCC. This is on x86_64 platform. Just running the fast tests. Look at test_ring_enqueue/test_ring_dequeue - those functions are large and branchy. The compiler is being forced to inline them. I'm not sure how many other places that's happening, but it simply cannot be correct. They're used all over that test_ring.c translation unit. That's a huge amount of expansion required by the compiler and a bunch of flow analysis being thrown away. You've pessimized for compilation memory. Can you please take a closer look? >> CC'd Honnappa (and others) to take a look. >> >> >> >> >> This is in a constrained (read: container) environment. I guess one >> >> way of resolving would be to allocate more memory to the container, >> >> but I'm also curious why the object files are getting so large? >> >> Should I consider this a bug or "working as intended"? This will >> >> have implications if we want asan/ubsan under the travis build also. >> >> >> >> -Aaron
> >> >> While compiling with asan and ubsan I run into the following error: > >> >> > >> >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o > >> >> > >> >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test > >> >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config > >> >> - > >> >> Ilib/librte_eal/include -I../lib/librte_eal/include > >> >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include > >> >> -Ilib/librte_eal/common - I../lib/librte_eal/common > >> >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include > >> >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - > >> >> I../lib/librte_kvargs -Ilib/librte_bitratestats > >> >> -I../lib/librte_bitratestats -Ilib/librte_ethdev > >> >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net > >> >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool > >> >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - > >> >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics > >> >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf > >> >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline > >> >> -I../lib/librte_cmdline -Ilib/librte_cryptodev > >> >> -I../lib/librte_cryptodev -Ilib/librte_distributor - > >> >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd > >> >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev > >> >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer > >> >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib > >> >> -I../lib/librte_rib -Ilib/librte_flow_classify - > >> >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table > >> >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched > >> >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag > >> >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci > >> >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - > >> >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security > >> >> -I../lib/librte_security -Ilib/librte_latencystats > >> >> -I../lib/librte_latencystats - Ilib/librte_member > >> >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline > >> >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - > >> >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder > >> >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - > >> >> I../drivers/mempool/ring -Idrivers/mempool/stack > >> >> -I../drivers/mempool/stack -Idrivers/event/skeleton > >> >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci > >> >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev > >> >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring > >> >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power > >> >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev > >> >> -I../lib/librte_compressdev -fdiagnostics-color=always > >> >> -fsanitize=address,undefined -fno-omit-frame-pointer -pipe > >> >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include > >> >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral > >> >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes > >> >> -Wnested-externs -Wold-style-definition -Wpointer-arith - > >> >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings > >> >> -Wno-missing-field-initializers -march=native -mno-avx512f - > >> >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - > >> MD -MQ > >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' > >> >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o > >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c > >> >> ../app/test/test_ring.c > >> >> > >> >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 > >> >> bytes > > I have a machine where I hit this error for a lot of other files every > > time I do a fresh compile. I attribute that to low swap space. > > I am shocked that a translation unit should be 4.6G? That's > unbelievably HUGE - there's no way that's correct. There must be > something going on with your machine - I've only ever seen it with > test_ring.c and it's very consistent. > > >> > > >> > I also noticed that test_ring.c compilation takes a huge amount of time and > >> memory. > >> > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still > > The compilation time issue was attributed to compiler version. We saw > > the issues only on clang. GCC did not show any problems. > > Is this problem something new? > > This is happening on GCC. Actually, the meson build with clang + > asan&ubsan doesn't even work right. > > >> seems too much. > >> > Will try to have a look later this week. > >> > >> Okay - glad I'm not the only one. My only theory is that all the inlining is > >> exploding the size of the file during the translation unit processing. > >> > >> I ran it through the preprocessor and didn't see any large arrays created > >> anywhere. But there are loads of calls to the rte_ring that are tagged as > >> "always inline" - even the test_enqueue and test_dequeue functions are > >> "always inline" and they are quite large. Just a completely unfounded guess. > > As a test, I changed the attributes in test_ring.h to remove the > always_inline - and I was finally able to complete a build with > asan&ubsan using GCC. This is on x86_64 platform. Just running the > fast tests. > > Look at test_ring_enqueue/test_ring_dequeue - those functions are large > and branchy. The compiler is being forced to inline them. I'm not sure > how many other places that's happening, but it simply cannot be > correct. They're used all over that test_ring.c translation unit. > That's a huge amount of expansion required by the compiler and a bunch > of flow analysis being thrown away. You've pessimized for compilation > memory. > > Can you please take a closer look? Can you try: http://patches.dpdk.org/patch/69559/ On my machine it takes less then 10s and 300MB after the patch. > > >> CC'd Honnappa (and others) to take a look. > >> > >> >> > >> >> This is in a constrained (read: container) environment. I guess one > >> >> way of resolving would be to allocate more memory to the container, > >> >> but I'm also curious why the object files are getting so large? > >> >> Should I consider this a bug or "working as intended"? This will > >> >> have implications if we want asan/ubsan under the travis build also. > >> >> > >> >> -Aaron
"Ananyev, Konstantin" <konstantin.ananyev@intel.com> writes: >> >> >> While compiling with asan and ubsan I run into the following error: >> >> >> >> >> >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o >> >> >> >> >> >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test >> >> >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config >> >> >> - >> >> >> Ilib/librte_eal/include -I../lib/librte_eal/include >> >> >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include >> >> >> -Ilib/librte_eal/common - I../lib/librte_eal/common >> >> >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include >> >> >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - >> >> >> I../lib/librte_kvargs -Ilib/librte_bitratestats >> >> >> -I../lib/librte_bitratestats -Ilib/librte_ethdev >> >> >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net >> >> >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool >> >> >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - >> >> >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics >> >> >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf >> >> >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline >> >> >> -I../lib/librte_cmdline -Ilib/librte_cryptodev >> >> >> -I../lib/librte_cryptodev -Ilib/librte_distributor - >> >> >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd >> >> >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev >> >> >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer >> >> >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib >> >> >> -I../lib/librte_rib -Ilib/librte_flow_classify - >> >> >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table >> >> >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched >> >> >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag >> >> >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci >> >> >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - >> >> >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security >> >> >> -I../lib/librte_security -Ilib/librte_latencystats >> >> >> -I../lib/librte_latencystats - Ilib/librte_member >> >> >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline >> >> >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - >> >> >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder >> >> >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - >> >> >> I../drivers/mempool/ring -Idrivers/mempool/stack >> >> >> -I../drivers/mempool/stack -Idrivers/event/skeleton >> >> >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci >> >> >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev >> >> >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring >> >> >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power >> >> >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev >> >> >> -I../lib/librte_compressdev -fdiagnostics-color=always >> >> >> -fsanitize=address,undefined -fno-omit-frame-pointer -pipe >> >> >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include >> >> >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral >> >> >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes >> >> >> -Wnested-externs -Wold-style-definition -Wpointer-arith - >> >> >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings >> >> >> -Wno-missing-field-initializers -march=native -mno-avx512f - >> >> >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - >> >> MD -MQ >> >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' >> >> >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o >> >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c >> >> >> ../app/test/test_ring.c >> >> >> >> >> >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 >> >> >> bytes >> > I have a machine where I hit this error for a lot of other files every >> > time I do a fresh compile. I attribute that to low swap space. >> >> I am shocked that a translation unit should be 4.6G? That's >> unbelievably HUGE - there's no way that's correct. There must be >> something going on with your machine - I've only ever seen it with >> test_ring.c and it's very consistent. >> >> >> > >> >> > I also noticed that test_ring.c compilation takes a huge amount of time and >> >> memory. >> >> > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still >> > The compilation time issue was attributed to compiler version. We saw >> > the issues only on clang. GCC did not show any problems. >> > Is this problem something new? >> >> This is happening on GCC. Actually, the meson build with clang + >> asan&ubsan doesn't even work right. >> >> >> seems too much. >> >> > Will try to have a look later this week. >> >> >> >> Okay - glad I'm not the only one. My only theory is that all the inlining is >> >> exploding the size of the file during the translation unit processing. >> >> >> >> I ran it through the preprocessor and didn't see any large arrays created >> >> anywhere. But there are loads of calls to the rte_ring that are tagged as >> >> "always inline" - even the test_enqueue and test_dequeue functions are >> >> "always inline" and they are quite large. Just a completely unfounded guess. >> >> As a test, I changed the attributes in test_ring.h to remove the >> always_inline - and I was finally able to complete a build with >> asan&ubsan using GCC. This is on x86_64 platform. Just running the >> fast tests. >> >> Look at test_ring_enqueue/test_ring_dequeue - those functions are large >> and branchy. The compiler is being forced to inline them. I'm not sure >> how many other places that's happening, but it simply cannot be >> correct. They're used all over that test_ring.c translation unit. >> That's a huge amount of expansion required by the compiler and a bunch >> of flow analysis being thrown away. You've pessimized for compilation >> memory. >> >> Can you please take a closer look? > > Can you try: > http://patches.dpdk.org/patch/69559/ > On my machine it takes less then 10s and 300MB after the patch. Sure! Thanks! >> >> >> CC'd Honnappa (and others) to take a look. >> >> >> >> >> >> >> >> This is in a constrained (read: container) environment. I guess one >> >> >> way of resolving would be to allocate more memory to the container, >> >> >> but I'm also curious why the object files are getting so large? >> >> >> Should I consider this a bug or "working as intended"? This will >> >> >> have implications if we want asan/ubsan under the travis build also. >> >> >> >> >> >> -Aaron
"Ananyev, Konstantin" <konstantin.ananyev@intel.com> writes: >> >> >> While compiling with asan and ubsan I run into the following error: >> >> >> >> >> >> FAILED: app/test/app@test@@dpdk-test@exe/test_ring.c.o >> >> >> >> >> >> gcc -Iapp/test/app@test@@dpdk-test@exe -Iapp/test -I../app/test >> >> >> -Ilib/librte_acl -I../lib/librte_acl -I. -I../ -Iconfig -I../config >> >> >> - >> >> >> Ilib/librte_eal/include -I../lib/librte_eal/include >> >> >> -Ilib/librte_eal/linux/include -I../lib/librte_eal/linux/include >> >> >> -Ilib/librte_eal/common - I../lib/librte_eal/common >> >> >> -Ilib/librte_eal/x86/include -I../lib/librte_eal/x86/include >> >> >> -Ilib/librte_eal -I../lib/librte_eal -Ilib/librte_kvargs - >> >> >> I../lib/librte_kvargs -Ilib/librte_bitratestats >> >> >> -I../lib/librte_bitratestats -Ilib/librte_ethdev >> >> >> -I../lib/librte_ethdev -Ilib/librte_net - I../lib/librte_net >> >> >> -Ilib/librte_mbuf -I../lib/librte_mbuf -Ilib/librte_mempool >> >> >> -I../lib/librte_mempool -Ilib/librte_ring -I../lib/librte_ring - >> >> >> Ilib/librte_meter -I../lib/librte_meter -Ilib/librte_metrics >> >> >> -I../lib/librte_metrics -Ilib/librte_bpf -I../lib/librte_bpf >> >> >> -Ilib/librte_cfgfile - I../lib/librte_cfgfile -Ilib/librte_cmdline >> >> >> -I../lib/librte_cmdline -Ilib/librte_cryptodev >> >> >> -I../lib/librte_cryptodev -Ilib/librte_distributor - >> >> >> I../lib/librte_distributor -Ilib/librte_efd -I../lib/librte_efd >> >> >> -Ilib/librte_hash -I../lib/librte_hash -Ilib/librte_eventdev >> >> >> -I../lib/librte_eventdev - Ilib/librte_timer -I../lib/librte_timer >> >> >> -Ilib/librte_fib -I../lib/librte_fib -Ilib/librte_rib >> >> >> -I../lib/librte_rib -Ilib/librte_flow_classify - >> >> >> I../lib/librte_flow_classify -Ilib/librte_table -I../lib/librte_table >> >> >> -Ilib/librte_port -I../lib/librte_port -Ilib/librte_sched >> >> >> -I../lib/librte_sched - Ilib/librte_ip_frag -I../lib/librte_ip_frag >> >> >> -Ilib/librte_kni -I../lib/librte_kni -Ilib/librte_pci >> >> >> -I../lib/librte_pci -Ilib/librte_lpm -I../lib/librte_lpm - >> >> >> Ilib/librte_ipsec -I../lib/librte_ipsec -Ilib/librte_security >> >> >> -I../lib/librte_security -Ilib/librte_latencystats >> >> >> -I../lib/librte_latencystats - Ilib/librte_member >> >> >> -I../lib/librte_member -Ilib/librte_pipeline -I../lib/librte_pipeline >> >> >> -Ilib/librte_rawdev -I../lib/librte_rawdev -Ilib/librte_rcu - >> >> >> I../lib/librte_rcu -Ilib/librte_reorder -I../lib/librte_reorder >> >> >> -Ilib/librte_stack -I../lib/librte_stack -Idrivers/mempool/ring - >> >> >> I../drivers/mempool/ring -Idrivers/mempool/stack >> >> >> -I../drivers/mempool/stack -Idrivers/event/skeleton >> >> >> -I../drivers/event/skeleton - Idrivers/bus/pci -I../drivers/bus/pci >> >> >> -I../drivers/bus/pci/linux -Idrivers/bus/vdev -I../drivers/bus/vdev >> >> >> -Idrivers/net/bonding - I../drivers/net/bonding -Idrivers/net/ring >> >> >> -I../drivers/net/ring -Ilib/librte_power -I../lib/librte_power >> >> >> -Ilib/librte_pdump -I../lib/librte_pdump -Ilib/librte_compressdev >> >> >> -I../lib/librte_compressdev -fdiagnostics-color=always >> >> >> -fsanitize=address,undefined -fno-omit-frame-pointer -pipe >> >> >> -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Werror -O2 -g -include >> >> >> rte_config.h -Wextra -Wcast-qual -Wdeprecated -Wformat- nonliteral >> >> >> -Wformat-security -Wmissing-declarations -Wmissing-prototypes >> >> >> -Wnested-externs -Wold-style-definition -Wpointer-arith - >> >> >> Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings >> >> >> -Wno-missing-field-initializers -march=native -mno-avx512f - >> >> >> DALLOW_EXPERIMENTAL_API -Wno-format-truncation -D_GNU_SOURCE - >> >> MD -MQ >> >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' >> >> >> -MF 'app/test/app@test@@dpdk-test@exe/test_ring.c.o.d' -o >> >> >> 'app/test/app@test@@dpdk-test@exe/test_ring.c.o' -c >> >> >> ../app/test/test_ring.c >> >> >> >> >> >> cc1: out of memory allocating 65536 bytes after a total of 4609626112 >> >> >> bytes >> > I have a machine where I hit this error for a lot of other files every >> > time I do a fresh compile. I attribute that to low swap space. >> >> I am shocked that a translation unit should be 4.6G? That's >> unbelievably HUGE - there's no way that's correct. There must be >> something going on with your machine - I've only ever seen it with >> test_ring.c and it's very consistent. >> >> >> > >> >> > I also noticed that test_ring.c compilation takes a huge amount of time and >> >> memory. >> >> > On my box it not as bad as yours (up to ~100 sec and ~1 GB) , but still >> > The compilation time issue was attributed to compiler version. We saw >> > the issues only on clang. GCC did not show any problems. >> > Is this problem something new? >> >> This is happening on GCC. Actually, the meson build with clang + >> asan&ubsan doesn't even work right. >> >> >> seems too much. >> >> > Will try to have a look later this week. >> >> >> >> Okay - glad I'm not the only one. My only theory is that all the inlining is >> >> exploding the size of the file during the translation unit processing. >> >> >> >> I ran it through the preprocessor and didn't see any large arrays created >> >> anywhere. But there are loads of calls to the rte_ring that are tagged as >> >> "always inline" - even the test_enqueue and test_dequeue functions are >> >> "always inline" and they are quite large. Just a completely unfounded guess. >> >> As a test, I changed the attributes in test_ring.h to remove the >> always_inline - and I was finally able to complete a build with >> asan&ubsan using GCC. This is on x86_64 platform. Just running the >> fast tests. >> >> Look at test_ring_enqueue/test_ring_dequeue - those functions are large >> and branchy. The compiler is being forced to inline them. I'm not sure >> how many other places that's happening, but it simply cannot be >> correct. They're used all over that test_ring.c translation unit. >> That's a huge amount of expansion required by the compiler and a bunch >> of flow analysis being thrown away. You've pessimized for compilation >> memory. >> >> Can you please take a closer look? > > Can you try: > http://patches.dpdk.org/patch/69559/ > On my machine it takes less then 10s and 300MB after the patch. Markedly improved - thanks! >> >> >> CC'd Honnappa (and others) to take a look. >> >> >> >> >> >> >> >> This is in a constrained (read: container) environment. I guess one >> >> >> way of resolving would be to allocate more memory to the container, >> >> >> but I'm also curious why the object files are getting so large? >> >> >> Should I consider this a bug or "working as intended"? This will >> >> >> have implications if we want asan/ubsan under the travis build also. >> >> >> >> >> >> -Aaron