[#4899] ethernet card would fail to function sometimes

Submitted By: Mingquan Pan

Open Date

2009-02-12 21:01:04 Close Date

2010-01-13 04:02:36

Priority:

Medium Assignee:

Mingquan Pan

Status:

Closed Fixed In Release:

2009R1-RC6

Found In Release:

2009R1-RC6 Release:

Category:

N/A Board:

N/A

Processor:

BF548 Silicon Revision:

Is this bug repeatable?:

Yes Resolution:

Not reproducible

Uboot version or rev.:

Toolchain version or rev.:

4.1 toolchain of Jan 16

App binary format:

N/A

Summary: ethernet card would fail to function sometimes

Details:

ethernet card on bf548 ezkit would fail to function sometimes. It looks if the board is left alone after booting up for an hour, it could be happening. Because in toolchain testing, after doing simulator test which is about an hour, and then it comes to flat testing, the board would be stuck at:

On host:

test 24376 0.0 0.0 2104 756 pts/3 S Feb12 0:00 rcp root 10.100.4.50 /bin/gdbserver /home/test/work/cruise/temp/regtest_build4.1/tests/har

postfix 25651 0.0 0.1 5836 1804 ? S 09:24 0:00 pickup -l -t fifo -u

the log is:

Targets: uclinux

Components: binutils gas ld gdb gcc g++ gfortranlibstdc++ libmudflap newlib

Host: i686-pc-linux-gnu

Build: i686-pc-linux-gnu

Found multiple versions of runtest, using the one at /usr/bin/runtest

Found multiple versions of bc, using the one at /usr/bin/bc

Found multiple versions of killall, using the one at /usr/bin/killall

Found multiple versions of pstree, using the one at /usr/bin/pstree

Found multiple versions of rsh, using the one at /usr/bin/rsh

Found multiple versions of rcp, using the one at /usr/bin/rcp

Toolchain source is at /home/test/work/cruise/checkouts/toolchain

Build will be done in /home/test/work/cruise/temp/regtest_build4.1

Path for test results

uclinux (10.100.4.50): /home/test/work/cruise/temp/regtest_build4.1/tests/hardware-flat

Creating log file /home/test/work/cruise/temp/regtest_build4.1/logs/regtest.log

On the target:

root:/> version

kernel: Linux release 2.6.28.3-ADI-2009R1-pre-svn6089, build #29 Thu Feb 12 14:18:53 CST 2009

toolchain: bfin-uclinux-gcc release gcc version 4.1.2 (ADI svn)

user-dist: release svn-7730, build #10 Thu Feb 12 14:17:41 CST 2009

root:/> ifconfig

eth0 Link encap:Ethernet HWaddr 00:E0:22:FE:BD:0A

inet addr:10.100.4.50 Bcast:10.100.4.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:72 errors:0 dropped:0 overruns:0 frame:0

TX packets:81 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:5093 (4.9 KiB) TX bytes:48920 (47.7 KiB)

Interrupt:175

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root:/> ping 10.100.4.174

PING 10.100.4.174 (10.100.4.174): 56 data bytes

--- 10.100.4.174 ping statistics ---

17 packets transmitted, 0 packets received, 100% packet loss

root:/> uptime

14:40:47 up 17:50, load average: 0.00, 0.00, 0.00

root:/> dmesg

Linux version 2.6.28.3-ADI-2009R1-pre-svn6089 (test@44-bf548-toolchain) (gcc version 4.1.2 (ADI svn)) #29 Thu Feb 12 14:18:53 CST 2009

console [early_BFuart0] enabled

early printk enabled on early_BFuart0

Board Memory: 64MB

Kernel Managed Memory: 64MB

Memory map:

fixedcode = 0x00000400-0x00000490

text = 0x00001000-0x001a9740

rodata = 0x001a9740-0x002378b4

bss = 0x002378c0-0x00249954

data = 0x00249954-0x00262000

stack = 0x00260000-0x00262000

init = 0x00262000-0x005b6000

available = 0x005b6000-0x03dff000

DMA Zone = 0x03e00000-0x04000000

Hardware Trace Active and Enabled

Boot Mode: 3

Compiled for ADSP-BF548 Rev 0.0

Warning: Compiled for Rev 0, but running on Rev 2

Blackfin Linux support by blackfin.uclinux.org/

Processor Speed: 525 MHz core clock and 131 MHz System Clock

boot memmap: 00000000005b6000 - 0000000003dff000 (usable)

On node 0 totalpages: 15871

free_area_init_node: node 0, pgdat 0025d6dc, node_mem_map 005bb000

DMA zone: 124 pages used for memmap

DMA zone: 0 pages reserved

DMA zone: 15747 pages, LIFO batch:3

Normal zone: 0 pages used for memmap

Movable zone: 0 pages used for memmap

NOMPU: setting up cplb tables

Instruction Cache Enabled for CPU0

Data Cache Enabled for CPU0 (write-through)

Built 1 zonelists in Zone order, mobility grouping off. Total pages: 15747

Kernel command line: root=/dev/mtdblock0 rw earlyprintk=serial,uart0,57600 panic=3 ip=10.100.4.50 ip=10.100.4.50:10.100.4.174:10.100.4.174:255.255.255.0:bf

Configuring Blackfin Priority Driven Interrupts

PID hash table entries: 256 (order: 8, 1024 bytes)

console handover: boot [early_BFuart0] -> real [ttyBF0]

Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)

Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)

Kernel managed physical pages: 15871

Memory available: 57056k/65536k RAM, (3408k init code, 1697k kernel code, 739k data, 2048k dma, 584k reserved)

Calibrating delay loop... 1046.52 BogoMIPS (lpj=2093056)

Security Framework initialized

Mount-cache hash table entries: 512

Blackfin Scratchpad data SRAM: 4 KB

Blackfin L1 Data A SRAM: 16 KB (15 KB free)

Blackfin L1 Data B SRAM: 16 KB (16 KB free)

Blackfin L1 Instruction SRAM: 48 KB (42 KB free)

Blackfin L2 SRAM: 128 KB (128 KB free)

PDA for CPU0 reserved at 00239178

net_namespace: 288 bytes

NET: Registered protocol family 16

Blackfin GPIO Controller

Blackfin DMA Controller

ezkit_init(): registering device resources

SCSI subsystem initialized

libata version 3.00 loaded.

usbcore: registered new interface driver usbfs

usbcore: registered new interface driver hub

usbcore: registered new device driver usb

musb_hdrc: version 6.0, musb-dma, host, debug=0

musb_hdrc: ConfigData=0x00 (UTMI-8)

musb_hdrc: HDRC RTL version 0.0

musb_hdrc: hw_ep 0shared, max 64

musb_hdrc: hw_ep 1tx, max 128

musb_hdrc: hw_ep 1rx, max 128

musb_hdrc: hw_ep 2tx, max 128

musb_hdrc: hw_ep 2rx, max 128

musb_hdrc: hw_ep 3tx, max 128

musb_hdrc: hw_ep 3rx, max 128

musb_hdrc: hw_ep 4tx, max 128

musb_hdrc: hw_ep 4rx, max 128

musb_hdrc: hw_ep 5tx, max 1024

musb_hdrc: hw_ep 5rx, max 1024

musb_hdrc: hw_ep 6tx, max 1024

musb_hdrc: hw_ep 6rx, max 1024

musb_hdrc: hw_ep 7tx, max 1024

musb_hdrc: hw_ep 7rx, max 1024

musb_hdrc: USB Host mode controller at ffc03c00 using DMA, IRQ 82

musb_hdrc musb_hdrc.0: MUSB HDRC host driver

musb_hdrc musb_hdrc.0: new USB bus registered, assigned bus number 1

usb usb1: configuration #1 chosen from 1 choice

hub 1-0:1.0: USB hub found

dma_alloc_init: dma_page @ 0x0066c000 - 512 pages at 0x03e00000

hub 1-0:1.0: 1 port detected

NET: Registered protocol family 2

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)

TCP established hash table entries: 2048 (order: 2, 16384 bytes)

TCP bind hash table entries: 2048 (order: 1, 8192 bytes)

TCP: Hash tables configured (established 2048 bind 2048)

TCP reno registered

NET: Registered protocol family 1

Setting up Blackfin MMR debugfs

msgmni has been set to 111

io scheduler noop registered

io scheduler anticipatory registered (default)

io scheduler cfq registered

bf54x-lq043: FrameBuffer initializing...

bfin-otp: initialized

Serial: Blackfin serial driver

bfin-uart.1: ttyBF0 at MMIO 0xffc02000 (irq = 48) is a BFIN-UART

brd: module loaded

smsc911x: Driver version 2007-07-13.

eth0: SMSC911x MAC Address: 00:e0:22:fe:bd:0a

Driver 'sd' needs updating - please use bus_type methods

Driver 'sr' needs updating - please use bus_type methods

scsi0 : pata-bf54x

ata1: PATA max UDMA/66 irq 68

ata1.00: ATA-7: FUJITSU MHW2040AT, 0000000B, max UDMA/100

ata1.00: 78140160 sectors, multi 16: LBA

ata1.00: configured for UDMA/66

blk_queue_max_hw_segments: set to minimum 1

scsi 0:0:0:0: Direct-Access ATA FUJITSU MHW2040A 0000 PQ: 0 ANSI: 5

sd 0:0:0:0: [sda] 78140160 512-byte hardware sectors: (40.0 GB/37.2 GiB)

sd 0:0:0:0: [sda] Write Protect is off

sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00

sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

sd 0:0:0:0: [sda] 78140160 512-byte hardware sectors: (40.0 GB/37.2 GiB)

sd 0:0:0:0: [sda] Write Protect is off

sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00

sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

sda: sda1

sd 0:0:0:0: [sda] Attached SCSI disk

physmap platform flash device: 02000000 at 20000000

physmap-flash.0: Found 1 x16 devices at 0x0 in 16-bit bank

physmap-flash.0: Found an alias at 0x1000000 for the chip at 0x0

Intel/Sharp Extended Query Table at 0x010A

Using buffer write method

Using auto-unlock on power-up/resume

cfi_cmdset_0001: Erase suspend on write enabled

erase region 0: offset=0x0,size=0x20000,blocks=127

erase region 1: offset=0xfe0000,size=0x8000,blocks=4

RedBoot partition parsing not available

Using physmap partition information

Creating 3 MTD partitions on "physmap-flash.0":

0x00000000-0x00040000 : "bootloader(nor)"

0x00040000-0x00440000 : "linux kernel(nor)"

0x00440000-0x01000000 : "file system(nor)"

bf5xx-nand bf5xx-nand.0: page_size=256, data_width=8, wr_dly=3, rd_dly=3

NAND device: Manufacturer ID: 0x20, Chip ID: 0xda (ST Micro NAND 256MiB 3,3V 8-bit)

Creating 2 MTD partitions on "NAND 256MiB 3,3V 8-bit":

0x00000000-0x00400000 : "linux kernel(nand)"

0x00400000-0x10000000 : "file system(nand)"

m25p80 spi0.1: m25p16 (2048 Kbytes)

Creating 2 MTD partitions on "m25p80":

0x00000000-0x00040000 : "bootloader(spi)"

0x00040000-0x00200000 : "linux kernel(spi)"

bfin-spi bfin-spi.0: Blackfin on-chip SPI Controller Driver, Version 1.0, regs_base@ffc00500, dma channel@4

bfin-spi bfin-spi.1: Blackfin on-chip SPI Controller Driver, Version 1.0, regs_base@ffc02300, dma channel@5

input: bf54x-keys as /devices/platform/bf54x-keys/input/input0

bf54x-keys: Blackfin BF54x Keypad registered IRQ 76

rtc-bfin rtc-bfin: rtc core: registered rtc-bfin as rtc0

i2c /dev entries driver

i2c-bfin-twi i2c-bfin-twi.0: Blackfin BF5xx on-chip I2C TWI Contoller, regs_base@ffc00700

i2c-bfin-twi i2c-bfin-twi.1: Blackfin BF5xx on-chip I2C TWI Contoller, regs_base@ffc02200

hid_zpff: Unknown symbol _input_ff_create_memless

usbcore: registered new interface driver usbhid

usbhid: v2.6:USB HID core driver

Advanced Linux Sound Architecture Driver Version 1.0.18rc3.

ASoC version 0.13.2

dma rx:0 tx:1, err irq:10, regs:ffc00800

AD1980 SoC Audio Codec

asoc: AC97 <-> bf5xx-ac97 mapping ok

ALSA device list:

#0: bf5xx-board (AD1980)

TCP cubic registered

NET: Registered protocol family 17

rtc-bfin rtc-bfin: setting system clock to 1970-01-28 20:50:29 UTC (2407829)

eth0: SMSC911x/921x identified at 0x24000000, IRQ: 175

eth0: link down

IP-Config: Complete:

device=eth0, addr=10.100.4.50, mask=255.255.255.0, gw=10.100.4.174,

host=bf548-ezkit, domain=, nis-domain=(none),

bootserver=10.100.4.174, rootserver=10.100.4.174, rootpath=

Freeing unused kernel memory: 3408k freed

eth0: link up, 100Mbps, full-duplex, lpa 0x41E1

Follow-ups

--- Mingquan Pan 2009-02-12 21:30:31

It looks this problem is not specific to Bf548, it is also found on bf527 ezkit

board. But when it fails to ping from the host, the bf548 is still active on

console, while bf527 is complete stuck and no response to key stroke at all.

est 3641 0.0 0.0 9928 580 ? S Jan15 0:04 sshd:

test@pts/2

test 3642 0.0 0.1 5052 1280 pts/2 Ss+ Jan15 0:03 -bash

test 9861 0.0 2.1 260288 21932 pts/2 Sl Feb12 0:49

/usr/lib/jvm/java-1.5.0-sun-1.5.0_update16/bin/java -cp

/usr/lib/jvm/java/lib/tools.jar:/h

root 9927 0.0 0.0 0 0 ? S Feb12 0:06 [pdflush]

root 9932 0.0 0.0 0 0 ? S Feb12 0:01 [pdflush]

test 9955 0.0 1.9 247476 20236 pts/2 Sl Feb12 0:30

/usr/lib/jvm/java-1.5.0-sun-1.5.0_update16/bin/java -classpath

/home/test/work/cruise/apac

test 9972 0.0 0.1 4256 1412 pts/2 S Feb12 0:00 /bin/sh

../../test_scripts/toolchain/toolchain_test

test 23571 1.7 4.0 113580 41036 ? SNl Feb10 69:28 beagled-helper

/usr/lib/beagle/IndexHelper.exe

postfix 27914 0.0 0.1 5836 1808 ? S 09:43 0:00 pickup -l -t

fifo -u

test 28011 97.9 0.0 7792 644 ? Rs Jan20 34348:21

bfin-linux-uclibc-gdb -nw -nx -i=mi

test 28371 0.0 0.0 2580 992 pts/1 R+ 10:24 0:00 ps aux

test 28899 0.0 0.1 4520 1748 pts/2 S Feb12 0:02 /bin/sh

./toolchain-regtest -s /home/test/work/cruise/checkouts/toolchain -o

/home/test/wo

test 30334 0.0 0.0 4520 804 pts/2 S Feb12 0:00 /bin/sh

./toolchain-regtest -s /home/test/work/cruise/checkouts/toolchain -o

/home/test/wo

test 30335 0.0 0.0 2092 740 pts/2 S Feb12 0:00 rsh -l root

10.100.4.50 ls / 1>/dev/null

test 30336 0.0 0.0 2896 576 pts/2 S Feb12 0:00 wc -l

test 30338 0.0 0.0 0 0 pts/2 Z Feb12 0:00 [rsh]

test@45-bf527-toolchain:~/work/cruise/test_scripts/toolchain/toolchain-build>

ping 10.100.4.50

PING 10.100.4.50 (10.100.4.50) 56(84) bytes of data.

--- 10.100.4.50 ping statistics ---

2 packets transmitted, 0 received, 100% packet loss, time 999ms

And this stuck on bf527 happens between the libstdc++ flat testing.

from the log, we can see:

PASS: 23_containers/deque/cons/clear_allocator.cc (test for excess errors)

Executing on bfin-uclinux: /tmp/clear_allocator.exe.21041 (timeout = 300)

Executing on bfin-uclinux: rm -f /tmp/clear_allocator.exe.21041 (timeout =

300)

Executed ./clear_allocator.exe, status 0

PASS: 23_containers/deque/cons/clear_allocator.cc execution test

extra_tool_flags are:

Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_co

ntainers/deque/explicit_instantiation/1.cc -S -mstack-check-l1 -o 1.s

(timeout = 300)

spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/

work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_containers/dequ

e/explicit_instantiation/1.cc -S -mstack-check-l1 -o 1.s^M

PASS: 23_containers/deque/explicit_instantiation/1.cc (test for excess errors)

extra_tool_flags are:

Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_co

ntainers/deque/explicit_instantiation/2.cc -S -mstack-check-l1 -o 2.s

(timeout = 300)

spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/

work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_containers/dequ

e/explicit_instantiation/2.cc -S -mstack-check-l1 -o 2.s^M

PASS: 23_containers/deque/explicit_instantiation/2.cc (test for excess errors)

extra_tool_flags are:

Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_co

ntainers/deque/explicit_instantiation/3.cc -S -mstack-check-l1 -o 3.s

(timeout = 300)

spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/

work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_containers/dequ

e/explicit_instantiation/3.cc -S -mstack-check-l1 -o 3.s^M

PASS: 23_containers/deque/explicit_instantiation/3.cc (test for excess errors)

extra_tool_flags are:

Executing on host: bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite

testsuite_abi.o testsuite_allocator.o testsuite_character.o testsuite_hooks.o /

home/test/work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_containers/deque/invalidation/1.cc

-mstack-check-l1 -Wl,-elf2flt=-s80000

-lm -o ./1.exe (timeout = 300)

spawn bfin-uclinux-g++ -g -O2 -D_GLIBCXX_ASSERT -fmessage-length=0

-DLOCALEDIR="/home/test/work/cruise/temp/bfin-uclinux/bin/../share/locale"

-I/home/test/

work/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite testsuite_abi.o

testsuite_allocator.o testsuite_character.o testsuite_hooks.o /home/test/wor

k/cruise/checkouts/toolchain/gcc-4.1/libstdc++-v3/testsuite/23_containers/deque/invalidation/1.cc

-mstack-check-l1 -Wl,-elf2flt=-s80000 -lm -o ./1.exe^M

PASS: 23_containers/deque/invalidation/1.cc (test for excess errors)

Download of ./1.exe to bfin-uclinux failed.

UNRESOLVED: 23_containers/deque/invalidation/1.cc execution test

extra_tool_flags are:

--- Michael Hennerich 2009-02-18 06:44:43

> It looks this problem is not specific to Bf548, it is also found on bf527

ezkit board.

I’ve been watching this bug a few days now.

Friday last week I setup a BF548-EZKIT as well as a BF527-EZKIT running the

entire weekend.

- Still responding to telnet on Monday morning.

On my end I’ve not seen such a bug since months.

I wonder if it is related to using crossover cables in conjunction with your

test Host machine.

I could imagine the Ethernet card enters some kind of power saving mode if

there is silence for more than 1h and fails to wakeup?

Or that the Ethernet outages are related to some broken testcases you’re

running overwriting kernel text or data?

Nevertheless I setup two boards running this simple script trying to simulate

your toolchain tests.

#!/bin/bash

c=1800;

echo Crash Test $c

while :

while [ $c -ne 0 ]

echo Crash Test $c

rcp /home/michael/devel/svn/uclinux-dist-trunk_3/romfs/bin/version

root@10.44.2.45:/bin/

rsh -l root 10.44.2.45 'date;md5sum /root/version;/root/version'

sleep 1;

c=`expr $c - 1`

done

sleep 3700

done

If it is still running tomorrow – I’ll do the same test with an X-over

cable to see if there are any differences.

-Michael

--- Michael Hennerich 2009-02-19 03:56:34

Both platforms running over night – no Network outages seen …

root:/> cat /proc/cpuinfo;uptime

processor : 0

vendor_id : Analog Devices

cpu family : 0x27de

model name : ADSP-BF548 525(MHz CCLK) 131(MHz SCLK) (mpu off)

stepping : 1

cpu MHz : 525.000/131.250000

bogomips : 1046.52

Calibration : 523264000 loops

cache size : 16 KB(L1 icache) 32 KB(L1 dcache-wb) 0 KB(L2 cache)

dbank-A/B : cache/cache

icache setup : 4 Sub-banks/4 Ways, 32 Lines/Way

dcache setup : 2 Super-banks/4 Sub-banks/2 Ways, 64 Lines/Way

L2 SRAM : 128KB

board name : ADI BF548-EZKIT

board memory : 65536 kB (0x00000000 -> 0x04000000)

kernel memory : 63480 kB (0x00001000 -> 0x03dff000)

21:28:21 up 21:28, load average: 0.00, 0.00, 0.00

root:/>

root:/> cat /proc/cpuinfo;uptime

processor : 0

vendor_id : Analog Devices

cpu family : 0x27e0

model name : ADSP-BF527 525(MHz CCLK) 131(MHz SCLK) (mpu off)

stepping : 0

cpu MHz : 525.000/131.250000

bogomips : 1046.52

Calibration : 523264000 loops

cache size : 16 KB(L1 icache) 32 KB(L1 dcache-wb) 0 KB(L2 cache)

dbank-A/B : cache/cache

icache setup : 4 Sub-banks/4 Ways, 32 Lines/Way

dcache setup : 2 Super-banks/4 Sub-banks/2 Ways, 64 Lines/Way

board name : ADI BF527-EZKIT

board memory : 65536 kB (0x00000000 -> 0x04000000)

kernel memory : 64504 kB (0x00001000 -> 0x03eff000)

22:12:11 up 20:04, load average: 0.45, 0.38, 0.26

root:/>

--- Michael Hennerich 2009-02-20 06:54:27

I run tests with a crossover cable, as well.

My little test script simulating your toolchain tests can run forever on my

side.

However I did notice that the first time booting I get u-boot network timeouts

(see 1.) with bad dropped packages. Once the board booted it will always boot

without any timeouts shown. (see 2.)

Once I reboot my Host machine I see the issue again. So this might be related

to the Auto-DIMX feature in my network card.

1.)

smc911x: initializing

smc911x: detected LAN9218 controller

smc911x: phy initialized

smc911x: MAC 00:e0:22:fe:b8:44

TFTP from server 192.168.1.1; our IP address is 192.168.1.45

Filename 'uImage'.

Load address: 0x1000000

Loading: smc911x: dropped bad packet. Status: 0x04e88026

smc911x: dropped bad packet. Status: 0x05488022

T smc911x: dropped bad packet. Status: 0x05a88022

T T smc911x: dropped bad packet. Status: 0x01248026

T T T T smc911x: dropped bad packet. Status: 0x01848026

T #################################################################

#################################################################

#######################

done

2.)

smc911x: initializing

smc911x: detected LAN9218 controller

smc911x: phy initialized

smc911x: MAC 00:e0:22:fe:b8:44

TFTP from server 192.168.1.1; our IP address is 192.168.1.45

Filename 'uImage'.

Load address: 0x1000000

Loading: #################################################################

#################################################################

#######################

done

On my end I can't recreate whatever you are seeing.

Can you please try two things?

1) On you test machine connect the failing boards via an Ethernet switch.

2) If 1) doesn't help try to run a known good toolchain test. Maybe from our

last release? It might me that some broken test cases screwing up the kernel.

-Michael

--- Mingquan Pan 2009-02-23 22:40:18

There is something I missing, would you extend the sleep time to 4 hours and

have a try?

Because it is found that this issue mostly happens on the machines that I use

toolchain-regtest doing the test instead of the ones I use BuildToolChain. The

difference between the two scripts in testing that I can tell is that the

testing sequence is changed. In BuildToolChain, it is testing as long as

building the three toolchains while the toolchain-regtest do the whole testing

after the building is finished completely. So the waiting time in

toolchain-regtest is much longer.

--- Mingquan Pan 2009-02-25 04:51:40

Now it is frequently stuck at :

test 17479 0.0 3.1 260800 31844 pts/2 Sl Feb16 11:50

/usr/lib/jvm/java-1.5.0-sun-1.5.0_update16/bin/java -cp

/usr/lib/jvm/java/lib/tools.jar:/h

test 17842 0.0 0.1 4524 1740 pts/2 S Feb24 0:00 /bin/sh

./toolchain-regtest -s /home/test/work/cruise/checkouts/toolchain -o

/home/test/wo

root 23008 0.0 0.0 0 0 ? S Feb24 0:03 [pdflush]

test 25687 0.0 0.0 4524 792 pts/2 S Feb24 0:00 /bin/sh

./toolchain-regtest -s /home/test/work/cruise/checkouts/toolchain -o

/home/test/wo

root 25688 0.0 0.0 2092 732 pts/2 S Feb24 0:00 rsh -l root

10.100.4.50 ls / 1>/dev/null

test 25689 0.0 0.0 2896 576 pts/2 S Feb24 0:00 wc -l

test 28197 0.0 0.2 61072 3052 ? S Feb19 0:04

/usr/bin/krunner_lock

test 28542 0.0 0.3 53260 3180 ? S Feb19 0:00 kblankscrn.kss

-root

root 31261 0.0 0.0 0 0 ? S Feb24 0:06 [pdflush]

test 31764 0.0 0.0 2056 840 pts/2 S+ 12:33 0:00 telnet

10.100.4.50

In this case, the stuck is related to the failure in gdb hardware testing which

has case make kernel reboot. But it looks that it should be happening before the

kernel reboot. Because if it is in u-boot or in the middle of booting up, rsh

should get : No route to host instead of being stuck.

--- Michael Hennerich 2009-02-25 05:12:47

>rsh should get : No route to host instead of being stuck.

Well - I don't see this.

my rsh call gets stuck in case I reset and boot into u-boot.

Do you still see this only with BF527-EZKIT and BF548-EZKIT?

-Michael

--- Mingquan Pan 2009-02-25 23:33:23

I see the rsh stuck also on bf537 stamp and bf561 ezkit during gdb testing.

And there is time kernel stuck happening during fdpic gfortran testing, which

is after gdb testing on bf561 ezkit.And in this testing, after gdb testing,that

is,before the fdpic gcc(including gfortran) testing, the kernel is alive with

good rsh action but having page allocation failure seeing from the

reset_board_log.

When stuck, from the console it prints:

BusyBJxuv1.13.2 (2009-02-25 16:04:30 CST) built-in shell (msh)mp to NULL

address

Deferred Exception context

CURRENT PROCESS:

COMM=rshd PID=22752

CPU = 0

TEXT = 0x00000000-0x00000000 DATA = 0x00000000-0x00000000

BSS = 0x00000000-0x00000000 USER-STACK = 0x00000000

return address: [0x00000000]; contents of:

SEQUENCER STATUS: Not tainted

SEQSTAT: 0006202d IPEND: 0030 SYSCFG: 0006

EXCAUSE : 0x2d

interrupts disabled

physical IVG5 asserted : <0xffa00be4> { _evt_ivhw + 0x0 }

RETE: <0x00000000> /* Maybe null pointer? */

RETN: <0x01c42000> /* kernel dynamic memory */

RETX: <0x00000480> /* Maybe fixed code section */

RETS: <0x00000000> /* Maybe null pointer? */

PC : <0x00000000> /* Maybe null pointer? */

DCPLB_FAULT_ADDR: <0x01898d78> [ inetd + 0x8d78 ]

ICPLB_FAULT_ADDR: <0x00000000> /* Maybe null pointer? */

PROCESSOR STATE:

R0 : ffffffff R1 : 0078e178 R2 : 0189775c R3 : 0000002f

R4 : 00000004 R5 : 00000006 R6 : 01897c04 R7 : 00000000

P0 : 0000000b P1 : 01897c04 P2 : 01897c18 P3 : 0078e178

P4 : 0078e1cd P5 : 01898f00 FP : 00000000 SP : 01c41f24

LB0: 0189278d LT0: 0189278a LC0: 00000000

LB1: 01891de9 LT1: 01891de8 LC1: 00000000

B0 : 00000000 L0 : 00000000 M0 : 00000000 I0 : 018975cb

B1 : 00000000 L1 : 00000000 M1 : 00000000 I1 : 01898f00

B2 : 00000000 L2 : 00000000 M2 : 00000000 I2 : 00000000

B3 : 00000000 L3 : 00000000 M3 : 00000000 I3 : 00000000

A0.w: 00000000 A0.x: 00000000 A1.w: 00000000 A1.x: 00000000

USP : 01898d7c ASTAT: 02002000

Hardware Trace:

0 Target : <0x00004a58> { _trap_c + 0x0 }

Source : <0xffa0066a> { _exception_to_level5 + 0x9e } CALL pcrel

1 Target : <0xffa005cc> { _exception_to_level5 + 0x0 }

Source : <0xffa0049a> { _bfin_return_from_exception + 0xe } RTX

2 Target : <0xffa0048c> { _bfin_return_from_exception + 0x0 }

Source : <0xffa00522> { _ex_trap_c + 0x66 } JUMP.S

3 Target : <0xffa004bc> { _ex_trap_c + 0x0 }

Source : <0xffa00704> { _trap + 0x38 } JUMP (P4)

4 Target : <0xffa006ea> { _trap + 0x1e }

Source : <0xffa006e6> { _trap + 0x1a } IF !CC JUMP

5 Target : <0xffa006cc> { _trap + 0x0 }

Source : <0x01893e6c> [ inetd + 0x3e6c ] RTS

6 Target : <0x01893e66> [ inetd + 0x3e66 ]

Source : <0x01893e7a> [ inetd + 0x3e7a ] JUMP.S

7 Target : <0x01893e72> [ inetd + 0x3e72 ]

Source : <0x018911a4> [ inetd + 0x11a4 ] RTS

8 Target : <0x01891194> [ inetd + 0x1194 ]

Source : <0x01893e6e> [ inetd + 0x3e6e ] CALL pcrel

9 Target : <0x01893e6e> [ inetd + 0x3e6e ]

Source : <0x01893e62> [ inetd + 0x3e62 ] IF !CC JUMP

10 Target : <0x01893e5a> [ inetd + 0x3e5a ]

Source : <0xffa00be0> { __common_int_entry + 0xd8 } RTI

11 Target : <0xffa00b7e> { __common_int_entry + 0x76 }

Source : <0xffa00da4> { _evt_system_call + 0x64 } JUMP.S

12 Target : <0xffa00da4> { _evt_system_call + 0x64 }

Source : <0xffa0088c> { _system_call + 0xb8 } RTS

13 Target : <0xffa00888> { _system_call + 0xb4 }

Source : <0xffa00878> { _system_call + 0xa4 } IF !CC JUMP

14 Target : <0xffa00872> { _system_call + 0x9e }

Source : <0xffa00862> { _system_call + 0x8e } IF !CC JUMP

15 Target : <0xffa00850> { _system_call + 0x7c }

Source : <0xffa00870> { _system_call + 0x9c } JUMP.S

Userspace Stack

Stack info:

SP: [0x01898d7c] <0x01898d7c> [ inetd + 0x8d7c ]

FP: (0x01898d9c)

Memory from 0x01898d70 to 01899000

01898d70: 00000000 00000000 00000000 [00000000] 00000000 00000000 0189278a

01891de8

01898d90: 0189278d 01891de9 00062000 (01898db0)<018910d6> 00000400

01898dc4 <0189118c>

01898db0:(01898de0)<01891120> 00000006 00000004 00000000 01898ddc

<018904a4> 00000000

01898dd0: 01898dd8 00000000 00000001 00000000 (01898f2c)<01890b24>

0078e14c 00000007

01898df0: 00000000 00000000 01898f20 01898f00 00000000 00000000 00000000

00000000

01898e10: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898e30: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898e50: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898e70: 00000000 00000000 00000000 00000000 00000070 00000000 00000000

00000000

01898e90: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898eb0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898ed0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000

00000000

01898ef0: 00000000 00000000 00000000 00000000 fe030002 ae04640a 00000000

00000000

01898f10: 17000002 00000000 00000000 00000000 00000001 00000000 00000010

(01898f5c)

01898f30:<01893e44> 01896084 01897544 01898fa0 00000000 01897c04

01896048 00000001

01898f50: 00000000 00000000 00000000 (00000000)<003c0ab8> 01896084

003d8020 003d8024

01898f70: 00000000 459865ba ffffffff 00000000 <018908e8> 00000000

00000000 01896048

01898f90: 0189605a 00000000 003e8ad2 00000001 01898fb4 00000000

<01898fc0> 01898fcd

01898fb0: 00000000 6962732f 6e692f6e 00647465 4d524554 6b6e753d 6e776f6e

54415000

01898fd0: 622f3d48 2f3a6e69 2f727375 3a6e6962 6962732f 752f3a6e 732f7273

006e6962

01898ff0: 6962732f 6e692f6e 00647465 00000000 0c443220

Return addresses in stack:

frame 1 : <0x018910d6> [ inetd + 0x10d6 ]

address : <0x0189118c> [ inetd + 0x118c ]

frame 2 : <0x01891120> [ inetd + 0x1120 ]

address : <0x018904a4> [ inetd + 0x4a4 ]

frame 3 : <0x01890b24> [ inetd + 0xb24 ]

frame 4 : <0x01893e44> [ inetd + 0x3e44 ]

frame 5 : <0x003c0ab8> [ init + 0xab8 ]

address : <0x018908e8> [ inetd + 0x8e8 ]

address : <0x01898fc0> [ inetd + 0x8fc0 ]

The whole info printed is attached.

At this point , on host:

root 31702 0.0 0.3 4052 1836 pts/4 S+ Feb25 0:00 bash

test 31731 0.0 0.6 8504 3344 pts/6 Ss+ Feb25 0:00 kermit

test 31783 0.0 0.3 4100 1744 pts/5 S Feb25 0:01 /bin/sh

./BuildToolChain -t -T 10.100.4.50 -j 1 -w -s

/home/test/work/cruise/checkouts/too

root 21678 0.0 0.0 0 0 ? S Feb25 0:05 [pdflush]

test 10017 0.0 0.6 7392 3304 ? Ss Feb25 0:00

bfin-uclinux-gdb -nw -nx

test 1364 0.0 0.2 3832 1220 pts/5 S 01:59 0:00 sh

test 1365 0.0 0.5 4900 2984 pts/5 S 01:59 0:00 make -k -j 1

check

test 5246 0.0 0.2 3576 1192 pts/5 S 03:16 0:00 /bin/sh -c

(rootme=`${PWDCMD-pwd}`; export rootme; srcdir=`cd /home/test/work/cruise/chec

test 5247 0.0 0.2 3836 1272 pts/5 S 03:16 0:00 /bin/sh -c

(rootme=`${PWDCMD-pwd}`; export rootme; srcdir=`cd /home/test/work/cruise/chec

test 5256 0.0 0.9 5824 4588 pts/5 S 03:16 0:05 expect --

/usr/share/dejagnu/runtest.exp --tool gfortran

root 19357 0.0 0.1 1936 580 pts/5 S 11:51 0:00 rcp

./assign.exe root 10.100.4.50 /tmp/assign.exe.5256

test 19358 0.0 0.0 1468 356 pts/5 S 11:51 0:00 cat

root 19415 0.0 0.4 8144 2232 ? Ss 11:53 0:00 sshd: test

[priv]

test 19420 0.0 0.4 8144 2340 ? S 11:54 0:00 sshd:

test@pts/10

test 19421 0.0 0.3 4312 1904 pts/10 Ss+ 11:54 0:00 -bash

test 19586 0.0 0.3 4100 1748 pts/5 S 12:03 0:00 /bin/sh

./BuildToolChain -t -T 10.100.4.50 -j 1 -w -s

/home/test/work/cruise/checkouts/too

root 19587 0.0 0.1 1924 568 pts/5 S 12:03 0:00 rsh -l root

10.100.4.50 ps

test 19588 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v ps

test 19589 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v /rshd

test 19590 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v \[

test 19591 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v

telnetd

test 19592 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v /init

test 19593 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v

inetd

test 19594 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v PID

test 19595 0.0 0.1 2696 696 pts/5 S 12:03 0:00 grep -v [

/]sh

test 19596 0.0 0.1 2696 692 pts/5 S 12:03 0:00 grep -v

/s*bin/.*d

test 19597 0.0 0.1 2692 672 pts/5 S 12:03 0:00 sed

/^[[:space:]]*$/d

postfix 19941 0.0 0.2 4740 1388 ? S 12:27 0:00 pickup -l -t

fifo -u

test 20102 0.0 0.1 2692 812 pts/5 R+ 12:38 0:00 ps aux

test@43-bf561-toolchain:~/work/cruise/test_scripts/toolchain/toolchain-build>

ping 10.100.4.50

PING 10.100.4.50 (10.100.4.50) 56(84) bytes of data.

64 bytes from 10.100.4.50: icmp_seq=1 ttl=64 time=0.160 ms

64 bytes from 10.100.4.50: icmp_seq=2 ttl=64 time=0.174 ms

64 bytes from 10.100.4.50: icmp_seq=3 ttl=64 time=0.161 ms

--- 10.100.4.50 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2001ms

rtt min/avg/max/mdev = 0.160/0.165/0.174/0.006 ms

test@43-bf561-toolchain:~/work/cruise/test_scripts/toolchain/toolchain-build>

telnet 10.100.4.50

Trying 10.100.4.50...

Connected to 10.100.4.50.

Escape character is '^]'.

ping can be through, telnet fails.

--- Michael Hennerich 2009-02-26 11:08:34

Grace,

This is definitely not a networking issue!!

The log attached includes the entire repertoire of possible Blackfin

exceptions.

Followed by 00-Memory dumps coming from rcpd, rshd, inted, sh etc.

So you see issues that only look like networking bugs on your host machine,

because the rshd, rcpd, and inetd running on Blackfin can’t allocate memory to

handle incoming requests.

The failing toolchain test cases are the problem in the first place.

They may allocate memory that isn’t freed after the testcase crashes.

So PLEASE assign this bug to whoever is responsible for the toolchain test

cases.

-Michael

--- Robin Getz 2009-02-26 13:06:39

Michael:

You said:

>They may allocate memory that isn’t freed after the testcase crashes.

that is still a kernel problem - the kernel needs to clean up/free all the

allocated resources when an application quits.

-Robin

--- Sonic Zhang 2009-02-26 23:07:18

It is also possible that the memory fragment issue is the root cause. Assign to

Grace till the toolchain test suite issue is solved.

--- Mingquan Pan 2010-01-13 04:02:27

This is an old issue that some case make kernel panic and thus stuck the

following cases. We now can log the output on target console and find the failed

case to fix. As to the memory allocate issue, it is something like the issue in

bug [5454], which is solved. Since the testing on trunk head is ok now and not

related to network, so close this bug.

Files

Changes

Commits

Dependencies

Duplicates

Associations

Partner Forums

Partner Libraries

[#4899] ethernet card would fail to function sometimes