FAQ:[#6057] Kernel crashed with slob configuration(2010)

Document created by Aaronwu Employee on Sep 11, 2013
Version 1Show Document
  • View in full screen mode

[#6057] Kernel crashed with slob configuration

Submitted By: Vivi Li

Open Date

2010-06-03 03:30:05    

Priority:

Medium     Assignee:

Bob Liu

Status:

Open     Fixed In Release:

N/A

Found In Release:

2010R1-RC5     Release:

Category:

N/A     Board:

EZBRD

Processor:

BF526     Silicon Revision:

Is this bug repeatable?:

Yes     Resolution:

Under Debugging

Uboot version or rev.:

    Toolchain version or rev.:

gcc4.3-2010_Apr_07

App binary format:

N/A     

Summary: Kernel crashed with slob configuration

Details:

 

Kernel crashed with slob configuration on BFIN526_EZBRD. Kernel config is attached.

 

Last passed version:

--

kernel:    Linux release 2.6.34-ADI-2010R1-pre-svn8831, build #80 Mon May 24 13:10:41 CST 2010

toolchain: bfin-uclinux-gcc release gcc version 4.3.4 (ADI-trunk/svn-3951)

user-dist: release svn-9641, build #1492 Mon May 24 13:09:55 CST 2010

--

 

First failed version:

--

kernel:    Linux release 2.6.34-ADI-2010R1-pre-svn8840, build #84 Tue May 25 21:20:21 CST 2010

toolchain: bfin-uclinux-gcc release gcc version 4.3.4 (ADI-trunk/svn-3951)

user-dist: release svn-9644, build #1570 Tue May 25 21:19:41 CST 2010

--

 

Bellow is the detailed log:

--

Linux version 2.6.34-ADI-2010R1-pre-svn8863 (test@uclinux81-bf526) (gcc version 4.3.4 (ADI-trunk/svn-3951) ) #104 Wed Jun 2 04:48:00 CST 2010

register early platform devices

bootconsole [early_shadow0] enabled

bootconsole [early_BFuart1] enabled

early printk enabled on early_BFuart1

Board Memory: 64MB

Kernel Managed Memory: 64MB

Memory map:

  fixedcode = 0x00000400-0x00000490

  text      = 0x00001000-0x00132458

  rodata    = 0x00132458-0x00198120

  bss       = 0x00199000-0x001a9d54

  data      = 0x001a9d54-0x001be000

    stack   = 0x001bc000-0x001be000

  init      = 0x001be000-0x0030c000

  available = 0x0030c000-0x03f00000

  DMA Zone  = 0x03f00000-0x04000000

Hardware Trace Active and Enabled

Boot Mode: 1

Blackfin support (C) 2004-2010 Analog Devices, Inc.

Compiled for ADSP-BF526 Rev 0.0

Blackfin Linux support by http://blackfin.uclinux.org/

Processor Speed: 400 MHz core clock and 80 MHz System Clock

NOMPU: setting up cplb tables

Instruction Cache Enabled for CPU0

  External memory: cacheable in instruction cache

Data Cache Enabled for CPU0

  External memory: cacheable (write-through) in data cache

Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 16002

Kernel command line: root=/dev/mtdblock0 rw clkin_hz=25000000 earlyprintk=serial,uart1,57600 console=ttyBF1,57600 ip=10.100.4.50:10.100.4.174:10.100.4.174:255.255.255.0:bf526-ezbrd:eth0:off

PID hash table entries: 256 (order: -2, 1024 bytes)

Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)

Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)

Memory available: 60824k/65536k RAM, (1336k init code, 1221k kernel code, 559k data, 1024k dma, 572k reserved)

Hierarchical RCU implementation.

NR_IRQS:159

Configuring Blackfin Priority Driven Interrupts

console [ttyBF1] enabled, bootconsole disabled

console [ttyBF1] enabled, bootconsole disabled

Calibrating delay loop... 792.57 BogoMIPS (lpj=1585152)

Security Framework initialized

Mount-cache hash table entries: 512

Blackfin Scratchpad data SRAM: 4 KB

Blackfin L1 Data A SRAM: 16 KB (16 KB free)

Blackfin L1 Data B SRAM: 16 KB (16 KB free)

Blackfin L1 Instruction SRAM: 48 KB (42 KB free)

NET: Registered protocol family 16

Blackfin DMA Controller

ezbrd_init(): registering device resources

bio: create slab <bio-0> at 0

NULL pointer access

Kernel OOPS in progress

Deferred Exception context

CURRENT PROCESS:

COMM=swapper PID=1  CPU=0

invalid mm

return address: [0x000404de]; contents of:

0x000404b0:  b371  323a  0a05  1451  e14c  001b  e10c  1914

0x000404c0:  0034  61f8  0040  9165  640d  c682  8ffd  67fd

0x000404d0:  6f45  6006  2003  3229  6f45  3255  6cc2 [a1a9]

0x000404e0:  0854  1826  0000  0000  0000  a928  09b8  1ff4

 

ADSP-BF526-0.0 400(MHz CCLK) 80(MHz SCLK) (mpu off)

Linux version 2.6.34-ADI-2010R1-pre-svn8863 (test@uclinux81-bf526) (gcc version 4.3.4 (ADI-trunk/svn-3951) ) #104 Wed Jun 2 04:48:00 CST 2010

 

SEQUENCER STATUS:               Not tainted

SEQSTAT: 00000027  IPEND: 8008  IMASK: 003f  SYSCFG: 0006

  Peripheral interrupts masked off

  Kernel interrupts masked off

  EXCAUSE   : 0x27

  physical IVG3 asserted : <0xffa00720> { _trap + 0x0 }

  physical IVG15 asserted : <0xffa00f5c> { _evt_system_call + 0x0 }

  logical irq   6 mapped  : <0xffa00358> { _bfin_coretmr_interrupt + 0x0 }

RETE: <0x00000000> /* Maybe null pointer? */

RETN: <0x02009ed8> /* kernel dynamic memory (maybe user-space) */

RETX: <0x00000480> /* Maybe fixed code section */

RETS: <0x000404fc> { _slob_alloc + 0x58 }

PC  : <0x000404de> { _slob_alloc + 0x3a }

DCPLB_FAULT_ADDR: <0x00000000> /* Maybe null pointer? */

ICPLB_FAULT_ADDR: <0x000404de> { _slob_alloc + 0x3a }

PROCESSOR STATE:

R0 : 00000000    R1 : 00000000    R2 : 0000000c    R3 : 0000000c

R4 : 0000ffff    R5 : 00000048    R6 : 00000000    R7 : 00000024

P0 : 02036000    P1 : 02035000    P2 : 00000000    P3 : 0034e758

P4 : 001b1914    P5 : ffffffe8    FP : 00000020    SP : 02009dfc

LB0: ffa0159c    LT0: ffa01596    LC0: 00000000

LB1: 0008dd38    LT1: 0008dd2c    LC1: 00000000

B0 : 00000000    L0 : 00000000    M0 : 0000002c    I0 : 00000fff

B1 : 00000000    L1 : 00000000    M1 : 00000000    I1 : 00000001

B2 : 00000000    L2 : 00000000    M2 : 00000000    I2 : ffffffe0

B3 : 00000000    L3 : 00000000    M3 : 00000000    I3 : 00000000

A0.w: 00000005   A0.x: 00000000   A1.w: 00000005   A1.x: 00000000

USP : 00000000  ASTAT: 00002022

 

Hardware Trace:

...

--

 

Follow-ups

 

--- Mike Frysinger                                           2010-06-03 13:04:07

i'm pretty sure we'll see same behavior with SLOB on all boards

 

--- Vivi Li                                                  2010-06-03 22:46:40

This bug first happens on May, 25th. And I haven't seen this bug on other plat

form yet.

 

--- Vivi Li                                                  2010-06-23 06:01:57

I also see it hang at "bio: create slab <bio-0> at 0" on

bf527-ezkit, though it doesn't happen all the time.

 

--- Bob Liu                                                  2010-07-08 02:55:05

I can also see it in the Last passed version and even 2.6.33 kernel version.

 

It seems that the page's memory was modified exceptionally.

 

 

--- Bob Liu                                                  2010-07-08 04:01:24

537-stamp have this problem if SCSI support is added in kernel config.

And I also think we can see it on all boards.

 

--- Vivi Li                                                  2011-04-29 03:15:57

Now I see this crash on slob on both bf527 and bf526.

 

--- Bob Liu                                                  2011-06-01 04:28:19

This bug should be fixed by commit:

http://blackfin.uclinux.org/git/?p=linux-kernel;a=commitdiff;h=e1066aa334b3155befa2779bad1028334dbe13fa

 

--- Vivi Li                                                  2011-06-08 01:39:14

It seems OK on bf526-ezbrd now. But I can still see slob kernel crash from time

to time, for example on regression machine 10.99.29.57 with bf537-stamp.

 

Bellow is the log:

--

Linux version 2.6.39-ADI-2011R1-pre-svn9919

(test@uclinux57-usb1761-vs6624-ad1371) (gcc version 4.3.5 (ADI-trunk/svn-5013) )

#12 Thu Jun 2 04:53:32 CST 2011

register early platform devices

bootconsole [early_shadow0] enabled

bootconsole [early_BFuart0] enabled

early printk enabled on early_BFuart0

Limiting kernel memory to 56MB due to anomaly 05000263

Board Memory: 64MB

Kernel Managed Memory: 64MB

Memory map:

  fixedcode = 0x00000400-0x00000490

  text      = 0x00001000-0x00117830

  rodata    = 0x00117830-0x0016e8b8

  bss       = 0x0016f000-0x0017f8d8

  data      = 0x0017f8d8-0x00190000

    stack   = 0x0018e000-0x00190000

  init      = 0x00190000-0x00312000

  available = 0x00312000-0x03800000

  DMA Zone  = 0x03f00000-0x04000000

Hardware Trace active and enabled

Boot Mode: 0

Blackfin support (C) 2004-2010 Analog Devices, Inc.

Compiled for ADSP-BF537 Rev 0.2

Blackfin Linux support by http://blackfin.uclinux.org/

Processor Speed: 500 MHz core clock and 125 MHz System Clock

NOMPU: setting up cplb tables

Instruction Cache Enabled for CPU0

  External memory: cacheable in instruction cache

Data Cache Enabled for CPU0

  External memory: cacheable (write-back) in data cache

Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 14224

Kernel command line: root=/dev/mtdblock0 rw clkin_hz=25000000

earlyprintk=serial,uart0,57600 console=ttyBF0,57600

ip=10.100.4.50:10.100.4.174:10.100.4.174:255.255.255.0:bf537-stamp:eth0:off

PID hash table entries: 256 (order: -2, 1024 bytes)

Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)

Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)

Memory available: 53592k/65536k RAM, (1544k init code, 1114k kernel code, 482k

data, 1024k dma, 7780k reserved)

NR_IRQS:146

Configuring Blackfin Priority Driven Interrupts

bfin-rtc: invalid date; resetting

console [ttyBF0] enabled, bootconsole disabled

console [ttyBF0] enabled, bootconsole disabled

Calibrating delay loop... 995.32 BogoMIPS (lpj=1990656)

pid_max: default: 32768 minimum: 301

Mount-cache hash table entries: 512

Blackfin Scratchpad data SRAM: 4 KB

Blackfin L1 Data A SRAM: 16 KB (16 KB free)

Blackfin L1 Data B SRAM: 16 KB (16 KB free)

Blackfin L1 Instruction SRAM: 48 KB (35 KB free)

NET: Registered protocol family 16

Blackfin DMA Controller

stamp_init(): registering device resources

bio: create slab <bio-0> at 0

bfin-spi bfin-spi.0: Blackfin on-chip SPI Controller Driver, Version 1.0,

regs_base@ffc00500, dma channel@7

Switching to clocksource bfin_cs_cycles

NET: Registered protocol family 2

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)

TCP established hash table entries: 2048 (order: 2, 16384 bytes)

TCP bind hash table entries: 2048 (order: 1, 8192 bytes)

TCP: Hash tables configured (established 2048 bind 2048)

TCP reno registered

UDP hash table entries: 256 (order: 0, 4096 bytes)

UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)

NET: Registered protocol family 1

debug-mmrs: setting up Blackfin MMR debugfs

msgmni has been set to 104

io scheduler noop registered (default)

bfin-uart: Blackfin serial driver

bfin-uart.0: ttyBF0 at MMIO 0xffc00400 (irq = 18) is a BFIN-UART

brd: module loaded

bfin_mii_bus: probed

bfin_mac: attached PHY driver [SMSC LAN83C185] (mii_bus:phy_addr=0:01, irq=-1,

mdc_clk=2500000Hz(mdc_div=24)@sclk=125MHz)

bfin_mac bfin_mac.0: eth0: Features changed: 0x00004800 -> 0x00004000

bfin_mac bfin_mac.0: eth0: Blackfin on-chip Ethernet MAC driver, Version 1.1

Jump to NULL address

Kernel OOPS in progress

Deferred Exception context

CURRENT PROCESS:

COMM=rcu_kthread PID=6  CPU=0

invalid mm

return address: [0x  (null)]; contents of:

 

ADSP-BF537-0.2 500(MHz CCLK) 125(MHz SCLK) (mpu off)

Linux version 2.6.39-ADI-2011R1-pre-svn9919

(test@uclinux57-usb1761-vs6624-ad1371) (gcc version 4.3.5 (ADI-trunk/svn-5013) )

#12 Thu Jun 2 04:53:32 CST 2011

 

SEQUENCER STATUS:               Not tainted

SEQSTAT: 0000002d  IPEND: 8008  IMASK: ffff  SYSCFG: 0006

  EXCAUSE   : 0x2d

  physical IVG3 asserted : <0xffa007b8> { _trap + 0x0 }

  physical IVG15 asserted : <0xffa0109c> { _evt_system_call + 0x0 }

  logical irq   6 mapped  : <0xffa003cc> { _bfin_coretmr_interrupt + 0x0

}

  logical irq 106 mapped  : <0x000bdda8> { _bfin_mac_interrupt + 0x0 }

RETE: <0x00000000> /* Maybe null pointer? */

RETN: <0x02017f68> /* kernel dynamic memory (maybe user-space) */

RETX: <0x00000480> /* Maybe fixed code section */

RETS: <0x00035c60> { _rcu_process_callbacks + 0x58 }

PC  : <0x00000000> /* Maybe null pointer? */

DCPLB_FAULT_ADDR: <0x023090e0> /* kernel dynamic memory (maybe

user-space) */

ICPLB_FAULT_ADDR: <0x00000000> /* Maybe null pointer? */

PROCESSOR STATE:

R0 : 023090dc    R1 : 00000200    R2 : 00000200    R3 : 0000016c

R4 : 00180e1c    R5 : 000241a4    R6 : 02016000    R7 : 00000000

P0 : 022ef380    P1 : 022ef000    P2 : 00000000    P3 : 02016000

P4 : 02017f8c    P5 : 00000000    FP : 02017f98    SP : 02017e8c

LB0: ffa01770    LT0: ffa0176e    LC0: 00000000

LB1: 0009524c    LT1: 00095240    LC1: 00000000

B0 : 0000001b    L0 : 00000000    M0 : ffffffff    I0 : 00000146

B1 : 00000000    L1 : 00000000    M1 : 00000000    I1 : 00000001

B2 : 00000000    L2 : 00000000    M2 : 00000000    I2 : fffffffc

B3 : 00000000    L3 : 00000000    M3 : 00000000    I3 : 0000000f

A0.w: 00000000   A0.x: 00000000   A1.w: 00000000   A1.x: 00000000

USP : 00000000  ASTAT: 00002000

 

Hardware Trace:

...

--

 

--- Vivi Li                                                  2011-08-22 05:56:52

I can still see this crash on bf537-stamp in buildroot.

To reproduce this bug we need to reboot kernel several times.

 

--- Vivi Li                                                  2011-08-30 02:10:19

Attach config in buildroot.

 

--- Vivi Li                                                  2011-10-19 04:49:53

Please try again use attached kernel config. Userspace config is not related and

you can use default user config.

I can always see crash with this config on bf537-stamp.

 

--- Tomasz Motylewski                                        2011-11-28 11:39:21

I also get this bug triggered on 2.6.39.1 - 2.6.39.3. It is very sensitive to

minute changes in kernel config and also in u-boot environment. It is also

sensitive to RAM contents before kernel boots (or whatever state preserved over

board reset).

 

when I symlink kernel (vmImage) as vmlinux-SVN-N3.img (nfsboot) it may crash,

while when I load the same kernel named vmlinux-SVN.img over nfsboot it works.

Then I change unrelated option and it reproducibly crashes (after CPU reset

pin). But when I then do hard power-down-up it suddenly runs OK. all with

u-boot-1.1.6.

BF537-stamp-like board.

 

BTW. With which bootloader do you use it? 1.1.6 has a bug - it leaves UART_IER

enabled, and when send serial char while kernel boots, it will crash after

ttyBF0 init. This I have fixed (UART_IER = 0) in my

native_machine_early_platform_add_devices() or stamp_init(). Whether if fixes

SLOB bug - not sure yet.

 

--- Sonic Zhang                                              2011-11-28 22:12:28

Please ask your uboot question on Analog EngineerZone uboot forum.

Please always use the same release version of uClinux-dist and u-boot.

For example, 2010R1 uClinux-dist and 2010R1 u-boot or trunk uClinux-dist and

trunk u-boot.

 

 

 

    Files

    Changes

    Commits

    Dependencies

    Duplicates

    Associations

    Tags

 

File Name     File Type     File Size     Posted By

No Files Were Found

Attachments

    Outcomes