FAQ: [#6333] Kgdb smp test fail in trunk and branch(2010)

Document created by Aaronwu Employee on Sep 11, 2013
Version 1Show Document
  • View in full screen mode

[#6333] Kgdb smp test fail in trunk and branch

Submitted By: Vivi Li

Open Date

2010-11-09 02:01:15     Close Date

2011-09-14 03:16:18

Priority:

Medium     Assignee:

Sonic Zhang

Status:

Closed     Fixed In Release:

N/A

Found In Release:

2010R1-RC5     Release:

Category:

N/A     Board:

N/A

Processor:

BF561     Silicon Revision:

Is this bug repeatable?:

Yes     Resolution:

Fixed

Uboot version or rev.:

    Toolchain version or rev.:

2010r1-rc4

App binary format:

N/A     

Summary: Kgdb smp test fail in trunk and branch

Details:

 

Kgdb smp test fail in trunk and branch. Config file is attached.

 

Last passed version:

--

kernel:    Linux release 2.6.34.1-ADI-2010R1-pre-svn9015, build #28 SMP Wed Jul 28 12:30:25 GMT 2010

toolchain: bfin-uclinux-gcc release gcc version 4.3.4 (ADI-trunk/svn-3951)

user-dist: release svn-9738, build #423 Wed Jul 28 12:29:13 GMT 2010

--

 

First failed version:

--

kernel:    Linux release 2.6.34.1-ADI-2010R1-pre-svn9020, build #32 SMP Thu Jul 29 14:25:43 GMT 2010

toolchain: bfin-uclinux-gcc release gcc version 4.3.4 (ADI-trunk/svn-3951)

user-dist: release svn-9742, build #478 Thu Jul 29 14:24:23 GMT 2010

--

 

Bellow is the error log:

--

Starting Kernel at = 001a6bac

console [ttyBF0] enabled, bootconsole disabled

Calibrating delay loop... 1187.84 BogoMIPS (lpj=2375680)

Mount-cache hash table entries: 512

CoreB bootstrap code to SRAM ff600000 via DMA.

Booting Core B.

Brought up 2 CPUs

Instruction Cache Enabled for CPU1

  External memory: cacheable in instruction cache

  L2 SRAM        : uncacheable in instruction cache

Data Cache Enabled for CPU1

  External memory: cacheable (write-through) in data cache

  L2 SRAM        : uncacheable in data cache

SMP: Total of 2 processors activated (4.09 BogoMIPS).

Blackfin Scratchpad data SRAM: 4 KB

Blackfin Scratchpad data SRAM: 4 KB

Blackfin L1 Data A SRAM: 16 KB (16 KB free)

Blackfin L1 Data A SRAM: 16 KB (16 KB free)

Blackfin L1 Data B SRAM: 16 KB (16 KB free)

Blackfin L1 Data B SRAM: 16 KB (16 KB free)

Blackfin L1 Instruction SRAM: 16 KB (15 KB free)

Blackfin L1 Instruction SRAM: 16 KB (15 KB free)

Blackfin L2 SRAM: 128 KB (127 KB free)

NET: Registered protocol family 16

Blackfin DMA Controller

ezkit_init(): registering device resources

bio: create slab <bio-0> at 0

bfin-spi bfin-spi.0: Blackfin on-chip SPI Controller Driver, Version 1.0, regs_base@ffc00500, dma channel@16

NET: Registered protocol family 2

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)

TCP established hash table entries: 2048 (order: 2, 16384 bytes)

TCP bind hash table entries: 2048 (order: 2, 16384 bytes)

TCP: Hash tables configured (established 2048 bind 2048)

TCP reno registered

UDP hash table entries: 128 (order: 0, 4096 bytes)

UDP-Lite hash table entries: 128 (order: 0, 4096 bytes)

NET: Registered protocol family 1

msgmni has been set to 110

io scheduler noop registered (default)

bfin-uart: Blackfin serial driver

bfin-uart.0: ttyBF0 at MMIO 0xffc00400 (irq = 35) is a BFIN-UART

brd: module loaded

smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@fluxnic.net>

eth0: SMC91C11xFD (rev 2) at 2c010300 IRQ 82 [nowait]

eth0: Ethernet addr: 00:e0:22:fe:ba:2a

console [netcon0] enabled

netconsole: network logging started

kgdboe: local port 6443

kgdboe: local IP 10.100.4.50

kgdboe: interface 'eth0'

kgdboe: remote port 6442

kgdboe: remote IP 10.100.4.174

kgdboe: remote ethernet address ff:ff:ff:ff:ff:ff

kgdboe: device eth0 not up yet, forcing it

eth0: link down

eth0: link up, 100Mbps, full-duplex, lpa 0x41E1

kgdb: Registered I/O driver kgdboe.

kgdboe: debugging over ethernet enabled

TCP cubic registered

NET: Registered protocol family 17

dma_alloc_init: dma_page @ 0x027b3000 - 256 pages at 0x03f00000

Freeing unused kernel memory: 5556k freed

                           _____________________________________

        a8888b.           / Welcome to the uClinux distribution \

       d888888b.         /       _     _                         \

       8P"YP"Y88        /       | |   |_|            __  __ (TM)  |

       8|o||o|88  _____/        | |    _ ____  _   _ \ \/ /       |

       8'    .88       \        | |   | |  _ \| | | | \  /        |

       8`._.' Y8.       \       | |__ | | | | | |_| | /  \        |

      d/      `8b.       \      \____||_|_| |_|\____|/_/\_\       |

     dP   .    Y8b.       \   For embedded processors including   |

    d8:'  "  `::88b        \    the Analog Devices Blackfin      /

   d8"         'Y88b        \___________________________________/

  :8P    '      :888

   8a.   :     _a88P         For further information, check out:

._/"Yaa_:   .| 88P|            - http://blackfin.uclinux.org/

\    YP"    `| 8P  `.          - http://docs.blackfin.uclinux.org/

/     \.___.d|    .'           - http://www.uclinux.org/

`--..__)8888P`._.'  jgs/a:f    - http://www.analog.com/blackfin

 

Have a lot of fun...

 

 

BusyBox v1.16.2 (2010-11-07 14:59:33 GMT) hush - the humble shell

 

root:/> ^[[6nversion

kernel:    Linux release 2.6.34.7-ADI-2010R1-svn9479, build #144 SMP Mon Nov 8 04:58:40 GMT 2010

toolchain: bfin-uclinux-gcc release gcc version 4.3.5 (ADI-2010R1-RC4)

user-dist: release svn-9892, build #1964 Mon Nov 8 04:57:25 GMT 2010

root:/> ^[[6nsuccessful boot attempt

************** STEP 3: Start KGDB TEST (SMP) (ethernet)

 

ifconfig eth0 10.100.4.50

root:/> ^[[6nspawn bfin-uclinux-gdb /home/test/work/cruise/checkouts/uclinux-dist/linux-2.6.x/vmlinux

^[[?1034hGNU gdb 6.6

Copyright (C) 2006 Free Software Foundation, Inc.

GDB is free software, covered by the GNU General Public License, and you are

welcome to change it and/or distribute copies of it under certain conditions.

Type "show copying" to see the conditions.

There is absolutely no warranty for GDB.  Type "show warranty" for details.

This GDB was configured as "--host=i686-pc-linux-gnu --target=bfin-uclinux"...

(gdb)

Case 1 ...PASS

 

Case 1 ...PASS

target remote udp:10.100.4.50:6443

warning: The remote protocol may be unreliable over UDP.

Some events may be lost, rendering further debugging impossible.

Remote debugging using udp:10.100.4.50:6443

 

Case 2 ...PASS

 

Case 2 ...PASS

kgdb_breakpoint ()

    at /home/test/work/cruise/checkouts/uclinux-dist/linux-2.6.x/arch/blackfin/include/asm/cache.h:56

56              __raw_smp_mark_barrier_asm();

(gdb) break kgdb_test

Breakpoint 1 at 0x8bf2: file arch/blackfin/kernel/kgdb_test.c, line 56.

 

Case 3 ...PASS

 

Case 3 ...PASS

(gdb) c

Continuing.

 

Case 4 ...PASS

 

Case 4 ...PASS

 

root:/> ^[[6n[New Thread 150]

[Switching to Thread 150]

 

Breakpoint 1, kgdb_test (name=0x11b55c "hello world!", len=12, count=85, z=16)

 

Case 5 ...PASS

 

Case 5 ...PASS

    at arch/blackfin/kernel/kgdb_test.c:56

56              pr_alert("kgdb name(%d): %s, %d, %d\n", len, name, count, z);

(gdb) hbreak 58

Hardware assisted breakpoint 2 at 0x8c18: file arch/blackfin/kernel/kgdb_test.c, line 58.

(gdb)

Case 6 ...PASS

 

Case 6 ...PASS

Num Type           Disp Enb Address    What

1   breakpoint     keep y   0x00008bf2 in kgdb_test

                                       at arch/blackfin/kernel/kgdb_test.c:56

        breakpoint already hit 1 time

2   hw breakpoint  keep y   0x00008c18 in kgdb_test

 

Case 7 ...PASS

 

Case 7 ...PASS

                                       at arch/blackfin/kernel/kgdb_test.c:58

(gdb) list

51      #endif

52

53

54      int kgdb_test(char *name, int len, int count, int z)

55      {

56              pr_alert("kgdb name(%d): %s, %d, %d\n", len, name, count, z);

57              count = z;

58              return count;

59      }

60

(gdb)

Case 8 ...PASS

 

Case 8 ...PASS

bt

#0  kgdb_test (name=0x11b55c "hello world!", len=12, count=85, z=16)

    at arch/blackfin/kernel/kgdb_test.c:56

 

Case 9 ...PASS

 

Case 9 ...PASS

#1  0x00008c48 in kgdb_test_proc_read (file=0x22fab40, buf=0x2a63e80 "",

    count=4096, ppos=0x2949eec) at arch/blackfin/kernel/kgdb_test.c:65

#2  0x0008023e in proc_reg_read (file=0x22fab40, buf=0x2a63e80 "", count=4096,

    ppos=0x2949eec) at fs/proc/inode.c:163

#3  0x00055a34 in vfs_read (file=0x22fab40, buf=0x2a63e80 "",

    count=<value optimized out>, pos=0x2949eec) at fs/read_write.c:293

#4  0x00055b5c in sys_read (fd=<value optimized out>, buf=0x2a63e80 "",

    count=4096) at fs/read_write.c:383

#5  0x0000bcea in system_call ()

    at /home/test/work/cruise/checkouts/uclinux-dist/linux-2.6.x/arch/blackfin/include/asm/cplb.h:128

(gdb) c

Continuing.

spawn /bin/sh

^[[?1034huclinux65-561-SMP:~/work/cruise/checkouts/uclinux-dist/testsuites/kgdb> ps aux |  grep bfin-uclinux-gdb

test      8972  1.3  2.7  34148 28332 pts/3    Ss+  05:00   0:00 bfin-uclinux-gdb /home/test/work/cruise/checkouts/uclinux-dist/linux-2.6.x/vmlinux

test      8983  0.0  0.0   5236   816 pts/4    R+   05:00   0:00 grep bfin-uclinux-gdb

uclinux65-561-SMP:~/work/cruise/checkouts/uclinux-dist/testsuites/kgdb> kill -9  `pidof  bfin-uclinux-gdb`

uclinux65-561-SMP:~/work/cruise/checkouts/uclinux-dist/testsuites/kgdb>

Last passed time: 2010_Jul_27_18_51, ucdist ver: 9738, kernel ver: 9015

--

 

Follow-ups

 

--- Mike Frysinger                                           2010-11-09 02:09:54

probably different issues.  trunk is tracked by [#6296].

 

--- Sonic Zhang                                              2010-11-10 00:19:36

After more investigation, I find the issue is not related to 32 bit or 64 bit

OS, but to which uClibc RPM is installed, default or full. On my 32 bit Suse

10.2, kernel binary built by toolchain with full uClibc can pass the kgdb

testing while that with default uClibc fails. And kernel binaries are also

different.

 

 

--- Vivi Li                                                  2010-11-10 05:43:28

The normal KGDB test on xenomai kernel will also fail at the same place.

 

--- Sonic Zhang                                              2010-11-12 00:55:53

KGDB single step in SMP kernel may hangs in flushinv without a csync

ahead. It looks the core internal write buffer should be flushed before

invalidating the data cache.

 

--- Vivi Li                                                  2010-11-15 02:12:41

Still fail in regression.

 

--- Vivi Li                                                  2010-11-15 22:14:05

Report KGDB test failure on xenomai kernel as a new bug 6341.

 

--- Sonic Zhang                                              2010-11-15 23:19:09

This bug is figured out to be silicon revision related. If the kernel is built

with silicon version 0.3 and running on v0.5 board, kgdb test passes. But, if it

is built with v0.5 and running on v0.5 board, test fails.

 

Looks a hardware anomaly is not worked around properly on bf561 v0.5 in SMP

kernel.

 

--- Sonic Zhang                                              2010-11-16 04:41:26

Now, the silicon version difference issue is narrowed down to anomaly 05000257.

 

 

"Hardware loops of length less than 4 instructions may cause the wrong

fetch address to be sent to the instruction fetch unit which will result in

unpredictable behavior.  This situation will arise when an interrupt or an

exception is generated at the end of a loop."

 

The recommended work around is

 

   R0 = LC0;

   LC0 = R0;

   R0 = LC1;

   LC1 = R0;

 

before return from exception.

 

Although it is said that this anomaly is fixed in bf561 v0.5, coreb hangs in

single step or trap exception in SMP kernel when running kgdb test. It looks UP

kernel doesn't affected by this bug.

 

I commit a temporary work around till this is confirmed by hardware team.

 

 

--- Vivi Li                                                  2010-11-18 04:24:25

Test pass after apply the work around.

 

--- Vivi Li                                                  2011-09-14 03:16:18

Close it.

 

 

 

    Files

    Changes

    Commits

    Dependencies

    Duplicates

    Associations

    Tags

 

File Name     File Type     File Size     Posted By

config.linux.kgdb_smp    application/octet-stream    30162    Vivi Li

config.config.kgdb_smp    application/octet-stream    49001    Vivi Li

Outcomes