[#5892] bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes
Submitted By: Mingquan Pan
Open Date
2010-02-08 05:39:22 Close Date
2012-05-15 02:39:33
Priority:
Medium High Assignee:
Vivi Li
Stuart Henderson
Board:
N/A Silicon Revision:
Resolution:
Fixed Fixed In Release:
N/A
Processor:
BF527
Host Operating System:
toolchain rev.:
trunk head 4.3 kernel rev.:
State:
Closed Found In Release:
Is this bug repeatable?:
Summary: bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes
Details:
bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes.
Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0 -DLOCALEDIR="." -I/home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/util /home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/23_containers/set/modifiers/16728.cc ./libtestc++.a -Wl,-elf2flt=-s80000 -lm -o ./16728.exe (timeout = 600)
spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0 -DLOCALEDIR="." -I/home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/util /home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/23_containers/set/modifiers/16728.cc ./libtestc++.a -Wl,-elf2flt=-s80000 -lm -o ./16728.exe^M
PASS: 23_containers/set/modifiers/16728.cc (test for excess errors)
Executing on bfin-uclinux: /tmp/16728.exe.17014 (timeout = 300)
Executing on bfin-uclinux: rm -f /tmp/16728.exe.17014 (timeout = 300)
Executed ./16728.exe, status 1
KILL
FAIL: 23_containers/set/modifiers/16728.cc execution test
extra_tool_flags are:
Follow-ups
--- Vivi Li 2010-07-01 03:11:36
This bug still exists.
--- Vivi Li 2010-07-08 04:59:39
It also may fail with bfin-linux-uclibc-libstdc++-4.3.
From the log on target board, I can see when this execution test fail, there
is
mem failure on target board, similar to bug [#5828].
--
16728.exe.2295 invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0^M
Mem-Info:^M
DMA per-cpu:^M
CPU 0: hi: 0, btch: 1 usd: 0^M
active_anon:0 inactive_anon:0 isolated_anon:0^M
active_file:8 inactive_file:33 isolated_file:0^M
unevictable:3975 dirty:0 writeback:0 unstable:0^M
free:1024 slab_reclaimable:244 slab_unreclaimable:408^M
mapped:0 shmem:0 pagetables:0 bounce:0^M
DMA free:4096kB min:4096kB low:5120kB high:6144kB active_anon:0kB
inactive_anon:0kB active_file:32kB inactive_file:132kB unevictable:15900kB
isolated(anon):0kB isolated(file):0kB present:56896kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:976kB
slab_unreclaimable:1632kB kernel_stack:160kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:540 all_unreclaimable? yes^M
lowmem_reserve[]: 0 0 0^M
DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB
1*4096kB 0*8192kB 0*16384kB 0*32768kB = 4096kB^M
3983 total pagecache pages^M
14336 pages RAM^M
544 pages reserved^M
1602 pages shared^M
11146 pages non-shared^M
Out of memory: kill process 666 (16728.exe.2295) score 454 or a child^M
Killed process 666 (16728.exe.2295) vsz:29064kB, anon-rss:0kB, file-rss:0kB^M
klogd invoked oom-killer: gfp_mask=0x44d0, order=0, oom_adj=0^M
free:1036 slab_reclaimable:244 slab_unreclaimable:408^M
DMA free:4144kB min:4096kB low:5120kB high:6144kB active_anon:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no^M
DMA: 2*4kB 1*8kB 2*16kB 2*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB
0*4096kB 0*8192kB 0*16384kB 0*32768kB = 4144kB^M
--- Stuart Henderson 2010-08-04 12:27:38
I couldn't reproduce this until I accidentally fired off two test runs at the
same time on the same board. We're running out of memory, but we really
shouldn't be with this test unless we're doing other things on the board we're
testing on at the same time. is this the case?
if not, do you have a consistent way of reproducing this problem?
--- Sonic Zhang 2010-08-16 03:10:18
Vivi reviews the test script to sequence test cases in the test rather than
running all cases concurrently.
--- Steve Kilbane 2010-08-23 03:55:27
I don't think Stuart meant that the test runs test cases concurrently - if
Stuart just kicks off a test run normally, this test was passing. The problem
was showing up when he (accidentally) started a test run on a board, when that
board was *already* running a different test run.
In other words, under normal circumstances, this test case passes for us, but
if the board's already under some kind of loading, it fails. So is this still
failing for you, and if so, what are the circumstances?
--- Mike Frysinger 2010-08-23 04:02:37
it isnt unusual for processes to get orphaned while testing, so i wouldnt be
surprised if that contributed to the sporadic failures. but i'm just guessing;
vivi/grace would know for sure of course.
--- Stuart Henderson 2010-09-06 06:30:28
I think what might be helpful for us is to better understand how you go about
testing.
Could you answer these questions:
Do you run each testsuite individually (e.g. toolchain-regtest gcc) or
all/multiple tests in a single batch?
When testing on hardware over the network, does each board get utilised by only
one test machine at a time? or is it possible multiple test machines could be
accessing a board at the same time?
How often are you seeing this failure?
Has it occurred recently?
--- Vivi Li 2010-09-06 22:55:33
each testsuite is run individually: gcc, g++, gfortran, libstdc++...
libstdc++ test is run after gfortran is finished.
The target board is only connected to one machine and can only be accessed by
that machine.
I can see this failure every month. On one machine, I see it happened 8 times
on Aug.
Recently it occurred on Sep, 1st.
--- Stuart Henderson 2010-09-08 09:44:58
So if i was to run:
buildscript/toolchain-regtest -t uclinux -T {ip} gcc
buildscript/toolchain-regtest -t uclinux -T {ip} g++
buildscript/toolchain-regtest -t uclinux -T {ip} gfortran
buildscript/toolchain-regtest -t uclinux -T {ip} libstdc++
I would effectively reproduce the order of testing that you are seeing the
problem with?
does it only occur on specific test machines?
if so, what distribution of linux are they running?
does it only occur on specific target boards?
if so, what kind?
thanks.
--- Steve Kilbane 2010-09-09 08:48:33
This one cropped up for me last night, on bfin-linux-uclibc, running all tests
on a BF548. What was different for me was the uImage: I built it using my
trampoline-enabled toolchain, and with the cacheflush syscall.
(I'm not blaming either of those things - they were just the reason for the
kernel rebuild. But what svn rev is your kernel source, Vivi?)
--- Vivi Li 2010-09-10 06:04:12
I always use latest source code to test. Last time I saw it on bf537 stamp on
Sep 1st, ucdist version is 9804, kernel version is 9118.
--- Stuart Henderson 2010-09-14 05:15:03
using the latest sources for the kernel and the toolchain, i'm still failing to
reproduce this.
--- Stuart Henderson 2010-10-06 09:04:50
i've (finally) reproduced this on a bf548. it seems the testsuite leaves loads
of files in the /root dir on the target board. over time, this grows to
>10MB when doing a full test run, reducing available memory and causing the
test to fall over. you can easily reproduce this by creating a 10Meg file:
> dd if=/dev/zero of=file.out bs=1MB count=10
copying it on to the target board and running the test. the test will fall
over as expected. delete the file and run again and we're ok.
i'm going to see if it's possible to add a clean out of /root at the start of
each testsuite.
--- Mike Frysinger 2010-10-06 09:55:59
why is it using /root and not /tmp ? the toolchain-regtest specifically takes
care of cleaning out /tmp ...
--- Stuart Henderson 2010-10-06 16:25:03
it seems the data files for libstdc++ get copied to /root. some tests appear to
require this to function currently as they access these files and rsh executes
from /root.
--- Mike Frysinger 2010-10-06 19:56:30
what if you set root's homedir to /tmp in /etc/passwd ? presumably it's copying
files based on that.
otherwise, the file deletion is in clean_hardware() ...
--- Stuart Henderson 2010-10-07 05:17:24
just adding /root to clean_hardware (and avoiding hidden files) seems the best
idea, as we've seen boards getting into bad states and rebooting midway through
testing before and if this would occur we'd get all sorts of issues with
inconsistent passwd and missing .rhosts files.
--- Steve Kilbane 2010-10-07 07:59:19
Meanwhile, I'm downgrading this one, as it's a testsuite issue again, not a
toolchain issue.
--- Stuart Henderson 2010-10-11 09:38:50
fixed on trunk.
--- Mingquan Pan 2012-05-15 03:39:35
Grep this case out from the logs of this year on bf537 stamp board, it all shows
pass. So close here.
Files
Changes
Commits
Dependencies
Duplicates
Associations
Tags
File Name File Type File Size Posted By
No Files Were Found