2010-07-15 10:49:19     Kernel hangs on BF531

2010-07-15 10:49:19     Kernel hangs on BF531

Rainer Kloud (AUSTRIA)

Message: 91313   

Hallo,

I have a problem with a custom board with BF531 and 16MB SDRAM. When I boot the board I get the error messages as seen below.

If I customize the kernel f.e. add NFS Support and the kernel size gets bigger (0,9MB instead of 0,82MB) the kernel hangs imediatelly after the output: Starting Kernel at ...

If I perform different customizations which results in different kernel sizes the kernel hangs on different places.

What can I do to find where my Problem is?

Thank you in advance!

Best Regards,

Rainer

## Booting kernel from Legacy Image at 00800000 ...                       

   Image Name:   Linux-2.6.28.10-ADI-2009R1.1                             

   Created:      2010-07-15  13:29:00 UTC                                 

   Image Type:   Blackfin Linux Kernel Image (gzip compressed)            

   Data Size:    861104 Bytes = 840.9 kB                                  

   Load Address: 00001000                                                 

   Entry Point:  0017f594                                                 

   Verifying Checksum ... OK                                              

   Uncompressing Kernel Image ... OK                                      

Starting Kernel at = 0017f594                                             

Linux version 2.6.28.10-ADI-2009R1.1 (rainer@atpc6mgclx) (gcc version 4.1.2 (ADI svn)) #147 Thu Jul 15 15:28:54 CEST 2010

bootconsole [early_shadow0] enabled                                                                                    

bootconsole [early_BFuart0] enabled                                                                                    

early printk enabled on early_BFuart0                                                                                  

Board Memory: 16MB                                                                                                     

Kernel Managed Memory: 16MB                                                                                            

Memory map:                                                                                                            

  fixedcode = 0x00000400-0x00000490                                                                                    

  text      = 0x00001000-0x00100d00                                                                                    

  rodata    = 0x00100d00-0x0014c038                                                                                    

  bss       = 0x0014d000-0x00165464                                                                                    

  data      = 0x00165480-0x00174000                                                                                    

    stack   = 0x00172000-0x00174000                                                                                    

  init      = 0x00174000-0x00190000                                                                                    

  available = 0x00190000-0x00eff000                                                                                    

  DMA Zone  = 0x00f00000-0x01000000                                                                                    

Hardware Trace Active and Enabled                                                                                      

Boot Mode: 2                                                                                                           

Blackfin support (C) 2004-2009 Analog Devices, Inc.                                                                    

Compiled for ADSP-BF531 Rev 0.5                                                                                        

Blackfin Linux support by   blackfin.uclinux.org/                                                                 

Processor Speed: 400 MHz core clock and 133 MHz System Clock                                                           

free_area_init -> start_mem is 0x190000 virtual_end is 0xeff000                                                        

NOMPU: setting up cplb tables                                                                                          

Instruction Cache Enabled for CPU0                                                                                     

Data Cache Enabled for CPU0 (write-back)                                                                               

Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 3809                                             

Kernel command line: root=/dev/mtdblock3 rw clkin_hz=40000000 earlyprintk=serial,uart0,57600 console=ttyBF0,57600      

Configuring Blackfin Priority Driven Interrupts                                                                        

PID hash table entries: 64 (order: 6, 256 bytes)                                                                       

console [ttyBF0] enabled, bootconsole disabled                                                                         

console [ttyBF0] enabled, bootconsole disabled                                                                         

Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)                                                           

Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)                                                            

Memory available: 13480k/16384k RAM, (112k init code, 1023k kernel code, 461k data, 1024k dma, 280k reserved)          

Calibrating delay loop... 796.67 BogoMIPS (lpj=1593344)                                                                

Security Framework initialized                                                                                         

Mount-cache hash table entries: 512                                                                                    

Blackfin Scratchpad data SRAM: 4 KB                                                                                    

Blackfin L1 Instruction SRAM: 16 KB (5 KB free)                                                                        

net_namespace: 288 bytes                                                                                               

NET: Registered protocol family 16                                                                                     

Blackfin DMA Controller                                                                                                

emic_cp6010_init(): registering device resources                                                                       

NET: Registered protocol family 2                                                                                      

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)                                                         

TCP established hash table entries: 512 (order: 0, 4096 bytes)                                                         

TCP bind hash table entries: 512 (order: -1, 2048 bytes)                                                               

TCP: Hash tables configured (established 512 bind 512)                                                                 

TCP reno registered                                                                                                    

NET: Registered protocol family 1                                                                                      

msgmni has been set to 26                                                                                              

io scheduler noop registered                                                                                           

io scheduler anticipatory registered (default)                                                                         

io scheduler cfq registered                                                                                            

Serial: Blackfin serial driver                                                                                         

bfin-uart.1: ttyBF0 at MMIO 0xffc00400 (irq = 21) is a BFIN-UART                                                       

brd: module loaded                                                                                                     

smsc911x: Driver version 2007-07-13.                                                                                   

eth0: SMSC911x MAC Address: 5e:5f:a8:e4:03:77                                                                          

physmap platform flash device: 00200000 at 20000000                                                                    

physmap-flash.0: Found 1 x16 devices at 0x0 in 16-bit bank                                                             

Amd/Fujitsu Extended Query Table at 0x0041                                                                            

physmap-flash.0: Swapping erase regions for broken CFI table.                                                          

number of CFI chips: 1                                                                                                 

cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness.                                               

RedBoot partition parsing not available                                                                                

Using physmap partition information                                                                                    

Creating 4 MTD partitions on "physmap-flash.0":                                                                        

0x00000000-0x00030000 : "uboot"                                                                                        

0x00030000-0x00040000 : "uboot-env"                                                                                    

0x00040000-0x00130000 : "linux kernel"                                                                                 

0x00130000-0x00200000 : "rootfs"                                                                                       

rtc-bfin rtc-bfin: rtc core: registered rtc-bfin as rtc0                                                               

bfin-wdt: initialized: timeout=20 sec (nowayout=0)                                                                     

TCP cubic registered                                                                                                   

NET: Registered protocol family 17                                                                                     

rtc-bfin rtc-bfin: setting system clock to 1975-07-15 14:21:28 UTC (174666088)                                         

VFS: Mounted root (cramfs filesystem) readonly.                                                                        

Freeing unused kernel memory: 112k freed                                                                               

dma_alloc_init: dma_page @ 0x00e36000 - 256 pages at 0x00f00000                                                        

NULL pointer access

Kernel OOPS in progress

Deferred Exception context

No Valid process in current context

return address: [0x0000dd16]; contents of:

0x0000dcf0:  0820  1868  0000  0000  0000  e408  004f  5010

0x0000dd00:  e608  004f  e408  002d  0c00  100d  0000  0000

0x0000dd10:  0000  e50a  008c [e512] 0024  0c42  1804  a050

0x0000dd20:  5010  b050  3046  4d68  4f68  3210  e120  00ff

ADSP-BF531-0.5 400(MHz CCLK) 133(MHz SCLK) (mpu off)

Double Fault

Kernel OOPS in progress

Deferred Exception context

No Valid process in current context

return address: [0x00003e24]; contents of:

0x00003e00:  e100  8acc  e300  6222  3046  4d68  4f68  3208

0x00003e10:  e140  0012  e100  8b14  e122  0086  61ff  914a

0x00003e20:  e512  008b [a051] b0f0  e140  0012  5091  e100

0x00003e30:  8af0  6421  e300  620a  e300  5f9a  3008  e140

ADSP-BF531-0.5 400(MHz CCLK) 133(MHz SCLK) (mpu off)

<NULL> version <NULL>

Built with gcc version 4.1.2 (ADI svn)

SEQUENCER STATUS:               Not tainted

SEQSTAT: 00060027  IPEND: 8078  SYSCFG: 0006

  EXCAUSE   : 0x27

  physical IVG3 asserted : <0xffa08684> { _trap + 0x0 }

  interrupts disabled

  physical IVG5 asserted : <0xffa08bcc> { _evt_ivhw + 0x0 }

  physical IVG6 asserted : <0xffa08cc8> { _evt_timer + 0x0 }

  physical IVG15 asserted : <0xffa08d34> { _evt_system_call + 0x0 }

  logical irq   6 mapped  : <0xffa0832c> { _timer_interrupt + 0x0 }

  logical irq  14 mapped  : <0x00000000> /* Maybe null pointer? */

  logical irq  21 mapped  : <0x00000000> /* Maybe null pointer? */

  logical irq  22 mapped  : <0x00000000> /* Maybe null pointer? */

  logical irq  37 mapped  : <0x00000000> /* Maybe null pointer? */

RETE: <0x00000000> /* Maybe null pointer? */

QuoteReplyEditDelete

2010-07-15 11:07:49     Re: Kernel hangs on BF531

Robin Getz (UNITED STATES)

Message: 91314   

Rainer:

Since we don't have any platforms like that - this might be a little tricky - if you have an extra - if you could dump in in the mail - I'll have a look...

For now...

https://docs.blackfin.uclinux.org/doku.php?id=linux-kernel:debug:post_mortem

What seems to be happening - is that you are getting a null pointer during kernel boot - which triggers the OOPS - but the OOPS also triggers a null pointer - which triggers a double fault...

Was that the entire message? There should have been a few more registers printed out by the double fault - what you pasted in looks incomplete.

-Robin

QuoteReplyEditDelete

2010-07-16 01:40:24     Re: Kernel hangs on BF531

Rainer Kloud (AUSTRIA)

Message: 91325    Dear Robin, thank you for your answer. The output which I have attached is complete. I get not more messages on the console. Unfortunately the bootarg option panic=XXX does not work and I have no reset. My kernel hangs forever. So I'm a little bit stranded ... Best Regards, Rainer

QuoteReplyEditDelete

2010-07-16 03:42:33     Re: Kernel hangs on BF531

Michael Hennerich (GERMANY)

Message: 91343    Using 16MB of SDRAM shouldn't cause random kernel crashes.

I would double check the u-boot SDRAM initialization

- as well as the kernel Kconfig defines used in mem_init.h (in case you use the CONFIG_BFIN_KERNEL_CLOCK option).

Try to run mtest (memory test) in u-boot.

As a second step slow down SCLK - let's say to 80MHz just to see if it makes a difference.

-Michael

QuoteReplyEditDelete

2010-07-16 07:47:53     Re: Kernel hangs on BF531

Rainer Kloud (AUSTRIA)

Message: 91344   

Hello,

I found out the following: The Board has not 16MB RAM, it has only 8MB of RAM. The 8MB are divided into 4 x 2MB with 2MB holes inbetween. The holes are mirrors of the previous 2 MB. So the Physical Layout is the following

2MB RAM Part0

2MB Hole (Mirror of RAM Part0)

2 MB RAM Part1

2MB Hole (Mirror of RAM Part1)

2MB RAM Part3

2MB Hole (Mirror of RAM Part3)

2MB RAM Part4

2MB Hole (Mirror of RAM Part4)

U-Boot has no Problem, because it needs not more then 2MB so it does not come over a 2MB Boundary. But the Linux Kernel damages itself when starting the init process.

How can I tell the Linux Kernel that the memory has holes?

Thank you in advance!

Best Regards,

Rainer

QuoteReplyEditDelete

2010-07-16 09:30:11     Re: Kernel hangs on BF531

Michael Hennerich (GERMANY)

Message: 91346    I don't think this is going to work.

Consider using 16MB of SDRAM

QuoteReplyEditDelete

2010-07-16 11:36:57     Re: Kernel hangs on BF531

Mike Frysinger (UNITED STATES)

Message: 91353   

please read the documentation:

  docs.blackfin.uclinux.org/doku.php?id=reduce_kernel_memory_footprint

you should however change your hardware by adding more RAM ... 8MB most likely is going to be a huge pain

QuoteReplyEditDelete

2010-07-16 11:37:38     Re: Kernel hangs on BF531

Mike Frysinger (UNITED STATES)

Message: 91354   

also, you need to fix your u-boot so that it programs the memory controller with the right settings.  it should not be configuring SDRAM to say it has 16MB.

QuoteReplyEditDelete

2010-07-20 17:10:29     Re: Kernel hangs on BF531

Robin Getz (UNITED STATES)

Message: 91514   

Rainer:

It might work (though with issues)...

Try defining memmap=nn[KMG]$ss[KMG] to mark specific memory as reserved (or do not use). For example to exclude memory from 0x18690000-0x1869ffff would be "memmap=64K$0x18690000"

-Robin

QuoteReplyEditDelete

2010-08-02 15:21:05     Re: Kernel hangs on BF531

Rainer Kloud (AUSTRIA)

Message: 92064   

Thank you Robin, it works!

I have added mem=16M and 4 times memmap=2M$0x..... to the kernel command line and after that I'm able to run uCLinux on my custom Board with 8MB RAM. Further I had to adapt the value of /proc/sys/vm/min_free_kbytes in /etc/rc (this is set there to 4MB which is too large for my system).

Best Regards,

Rainer