[#5892] bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes

Submitted By: Mingquan Pan

Open Date

2010-02-08 05:39:22 Close Date

2012-05-15 02:39:33

Priority:

Medium High Assignee:

Vivi Li

Stuart Henderson

Board:

N/A Silicon Revision:

Resolution:

Fixed Fixed In Release:

N/A

Processor:

BF527

Host Operating System:

toolchain rev.:

trunk head 4.3 kernel rev.:

State:

Closed Found In Release:

N/A

Is this bug repeatable?:

N/A

Summary: bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes

Details:

bfin-uclinux-libstdc++-4.3 23_containers/set/modifiers/16728.cc fail sometimes.

Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0 -DLOCALEDIR="." -I/home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/util /home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/23_containers/set/modifiers/16728.cc ./libtestc++.a -Wl,-elf2flt=-s80000 -lm -o ./16728.exe (timeout = 600)

spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0 -DLOCALEDIR="." -I/home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/util /home/test/work/cruise/checkouts/toolchain/gcc-4.3/libstdc++-v3/testsuite/23_containers/set/modifiers/16728.cc ./libtestc++.a -Wl,-elf2flt=-s80000 -lm -o ./16728.exe^M

PASS: 23_containers/set/modifiers/16728.cc (test for excess errors)

Executing on bfin-uclinux: /tmp/16728.exe.17014 (timeout = 300)

Executing on bfin-uclinux: rm -f /tmp/16728.exe.17014 (timeout = 300)

Executed ./16728.exe, status 1

KILL

FAIL: 23_containers/set/modifiers/16728.cc execution test

extra_tool_flags are:

Follow-ups

--- Vivi Li 2010-07-01 03:11:36

This bug still exists.

--- Vivi Li 2010-07-08 04:59:39

It also may fail with bfin-linux-uclibc-libstdc++-4.3.

From the log on target board, I can see when this execution test fail, there

mem failure on target board, similar to bug [#5828].

16728.exe.2295 invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0^M

Mem-Info:^M

DMA per-cpu:^M

CPU 0: hi: 0, btch: 1 usd: 0^M

active_anon:0 inactive_anon:0 isolated_anon:0^M

active_file:8 inactive_file:33 isolated_file:0^M

unevictable:3975 dirty:0 writeback:0 unstable:0^M

free:1024 slab_reclaimable:244 slab_unreclaimable:408^M

mapped:0 shmem:0 pagetables:0 bounce:0^M

DMA free:4096kB min:4096kB low:5120kB high:6144kB active_anon:0kB

inactive_anon:0kB active_file:32kB inactive_file:132kB unevictable:15900kB

isolated(anon):0kB isolated(file):0kB present:56896kB mlocked:0kB dirty:0kB

writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:976kB

slab_unreclaimable:1632kB kernel_stack:160kB pagetables:0kB unstable:0kB

bounce:0kB writeback_tmp:0kB pages_scanned:540 all_unreclaimable? yes^M

lowmem_reserve[]: 0 0 0^M

DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB

1*4096kB 0*8192kB 0*16384kB 0*32768kB = 4096kB^M

3983 total pagecache pages^M

14336 pages RAM^M

544 pages reserved^M

1602 pages shared^M

11146 pages non-shared^M

Out of memory: kill process 666 (16728.exe.2295) score 454 or a child^M

Killed process 666 (16728.exe.2295) vsz:29064kB, anon-rss:0kB, file-rss:0kB^M

klogd invoked oom-killer: gfp_mask=0x44d0, order=0, oom_adj=0^M

Mem-Info:^M

DMA per-cpu:^M

CPU 0: hi: 0, btch: 1 usd: 0^M

active_anon:0 inactive_anon:0 isolated_anon:0^M

active_file:8 inactive_file:33 isolated_file:0^M

unevictable:3975 dirty:0 writeback:0 unstable:0^M

free:1036 slab_reclaimable:244 slab_unreclaimable:408^M

mapped:0 shmem:0 pagetables:0 bounce:0^M

DMA free:4144kB min:4096kB low:5120kB high:6144kB active_anon:0kB

inactive_anon:0kB active_file:32kB inactive_file:132kB unevictable:15900kB

isolated(anon):0kB isolated(file):0kB present:56896kB mlocked:0kB dirty:0kB

writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:976kB

slab_unreclaimable:1632kB kernel_stack:160kB pagetables:0kB unstable:0kB

bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no^M

lowmem_reserve[]: 0 0 0^M

DMA: 2*4kB 1*8kB 2*16kB 2*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB

0*4096kB 0*8192kB 0*16384kB 0*32768kB = 4144kB^M

3983 total pagecache pages^M

--- Stuart Henderson 2010-08-04 12:27:38

I couldn't reproduce this until I accidentally fired off two test runs at the

same time on the same board. We're running out of memory, but we really

shouldn't be with this test unless we're doing other things on the board we're

testing on at the same time. is this the case?

if not, do you have a consistent way of reproducing this problem?

--- Sonic Zhang 2010-08-16 03:10:18

Vivi reviews the test script to sequence test cases in the test rather than

running all cases concurrently.

--- Steve Kilbane 2010-08-23 03:55:27

I don't think Stuart meant that the test runs test cases concurrently - if

Stuart just kicks off a test run normally, this test was passing. The problem

was showing up when he (accidentally) started a test run on a board, when that

board was *already* running a different test run.

In other words, under normal circumstances, this test case passes for us, but

if the board's already under some kind of loading, it fails. So is this still

failing for you, and if so, what are the circumstances?

--- Mike Frysinger 2010-08-23 04:02:37

it isnt unusual for processes to get orphaned while testing, so i wouldnt be

surprised if that contributed to the sporadic failures. but i'm just guessing;

vivi/grace would know for sure of course.

--- Stuart Henderson 2010-09-06 06:30:28

I think what might be helpful for us is to better understand how you go about

testing.

Could you answer these questions:

Do you run each testsuite individually (e.g. toolchain-regtest gcc) or

all/multiple tests in a single batch?

When testing on hardware over the network, does each board get utilised by only

one test machine at a time? or is it possible multiple test machines could be

accessing a board at the same time?

How often are you seeing this failure?

Has it occurred recently?

--- Vivi Li 2010-09-06 22:55:33

each testsuite is run individually: gcc, g++, gfortran, libstdc++...

libstdc++ test is run after gfortran is finished.

The target board is only connected to one machine and can only be accessed by

that machine.

I can see this failure every month. On one machine, I see it happened 8 times

on Aug.

Recently it occurred on Sep, 1st.

--- Stuart Henderson 2010-09-08 09:44:58

So if i was to run:

buildscript/toolchain-regtest -t uclinux -T {ip} gcc

buildscript/toolchain-regtest -t uclinux -T {ip} g++

buildscript/toolchain-regtest -t uclinux -T {ip} gfortran

buildscript/toolchain-regtest -t uclinux -T {ip} libstdc++

I would effectively reproduce the order of testing that you are seeing the

problem with?

does it only occur on specific test machines?

if so, what distribution of linux are they running?

does it only occur on specific target boards?

if so, what kind?

thanks.

--- Steve Kilbane 2010-09-09 08:48:33

This one cropped up for me last night, on bfin-linux-uclibc, running all tests

on a BF548. What was different for me was the uImage: I built it using my

trampoline-enabled toolchain, and with the cacheflush syscall.

(I'm not blaming either of those things - they were just the reason for the

kernel rebuild. But what svn rev is your kernel source, Vivi?)

--- Vivi Li 2010-09-10 06:04:12

I always use latest source code to test. Last time I saw it on bf537 stamp on

Sep 1st, ucdist version is 9804, kernel version is 9118.

--- Stuart Henderson 2010-09-14 05:15:03

using the latest sources for the kernel and the toolchain, i'm still failing to

reproduce this.

--- Stuart Henderson 2010-10-06 09:04:50

i've (finally) reproduced this on a bf548. it seems the testsuite leaves loads

of files in the /root dir on the target board. over time, this grows to

>10MB when doing a full test run, reducing available memory and causing the

test to fall over. you can easily reproduce this by creating a 10Meg file:

> dd if=/dev/zero of=file.out bs=1MB count=10

copying it on to the target board and running the test. the test will fall

over as expected. delete the file and run again and we're ok.

i'm going to see if it's possible to add a clean out of /root at the start of

each testsuite.

--- Mike Frysinger 2010-10-06 09:55:59

why is it using /root and not /tmp ? the toolchain-regtest specifically takes

care of cleaning out /tmp ...

--- Stuart Henderson 2010-10-06 16:25:03

it seems the data files for libstdc++ get copied to /root. some tests appear to

require this to function currently as they access these files and rsh executes

from /root.

--- Mike Frysinger 2010-10-06 19:56:30

what if you set root's homedir to /tmp in /etc/passwd ? presumably it's copying

files based on that.

otherwise, the file deletion is in clean_hardware() ...

--- Stuart Henderson 2010-10-07 05:17:24

just adding /root to clean_hardware (and avoiding hidden files) seems the best

idea, as we've seen boards getting into bad states and rebooting midway through

testing before and if this would occur we'd get all sorts of issues with

inconsistent passwd and missing .rhosts files.

--- Steve Kilbane 2010-10-07 07:59:19

Meanwhile, I'm downgrading this one, as it's a testsuite issue again, not a

toolchain issue.

--- Stuart Henderson 2010-10-11 09:38:50

fixed on trunk.

--- Mingquan Pan 2012-05-15 03:39:35

Grep this case out from the logs of this year on bf537 stamp board, it all shows

pass. So close here.

Files

Changes

Commits

Dependencies

Duplicates

Associations