2009-06-17 11:34:00 Reg: Static Linking of uclibc lib to track heap area corruption
Jayaprabhu Nadarajan (INDIA)
Message: 75901
Hi,
While running my application using the 2008R1.5 Kernel,in some scenarios i am getting a system crash. Please find below the crash dump. The updates of the analysis done are also below.
Data access misaligned address violation
- Attempted misaligned data memory or data cache access.
Defered Exception context
CURRENT PROCESS:
COMM=Application PID=107
TEXT = 0x03000000-0x032549ac DATA = 0x020009ac-0x02cc93e4
BSS = 0x02cc93e4-0x02ce0000 USER-STACK = 0x02cffe60
return address: [0x0332eb84]; contents of:
0x0332eb60: 51fa 181d 0000 0000 ac51 0c41 1809 0000
0x0332eb70: 0000 9109 3071 520e 6460 0803 1820 322a
0x0332eb80: 0c40 5a95 [bc51] bc90 1818 bc42 0c41 1802
0x0332eb90: bc8a 324a 9317 2fd3 6800 2fc4 0c40 1bfa
SEQUENCER STATUS: Tainted: P
SEQSTAT: 00060024 IPEND: 0030 SYSCFG: 0006
HWERRCAUSE: 0x18
EXCAUSE : 0x24
RETE: <0x00000000> /* Maybe null pointer? */
RETN: <0x005ec000> [ Application + 0x0 ]
RETX: <0x0332eb84> [ /lib/libuClibc-0.9.29.so + 0x2eb84 ]
RETS: <0x0332e760> [ /lib/libuClibc-0.9.29.so + 0x2e760 ]
PC : <0x0332eb84> [ /lib/libuClibc-0.9.29.so + 0x2eb84 ]
DCPLB_FAULT_ADDR: <0x03e802e4> /* unknown address */
ICPLB_FAULT_ADDR: <0x0018312c> { _decode_address + 0xf0 }
PROCESSOR STATE:
R0 : 01488000 R1 : 000014a8 R2 : 0a74630e R3 : 0bb0230e
R4 : 02cd1bd8 R5 : 02cd1514 R6 : 0148949c R7 : 0a749516
P0 : 01288ff4 P1 : 0148949c P2 : 0bb06302 P3 : 02cd1510
P4 : 00000000 P5 : 0a74630e FP : 005f88b8 SP : 005ebf24
LB0: 02dfd965 LT0: 02dfd958 LC0: 00000000
LB1: 0331cf7b LT1: 0331cf7a LC1: 00000000
B0 : 00000003 L0 : 00000000 M0 : 00000007 I0 : 02c18214
B1 : 00000016 L1 : 00000000 M1 : 00000000 I1 : 00000000
B2 : 02ec0004 L2 : 00000000 M2 : 00000000 I2 : 00000000
B3 : 00000140 L3 : 00000000 M3 : 000000ff I3 : 005f7720
A0.w: 00000000 A0.x: 00000000 A1.w: 00000000 A1.x: 00000000
USP : 005f88b8 ASTAT: 02003004
Hardware Trace:
0 Target : <0x00183a14> { _trap_c + 0x0 }
Source : <0xffa00734> { _exception_to_level5 + 0xb4 }
1 Target : <0xffa00680> { _exception_to_level5 + 0x0 }
Source : <0xffa005d8> { _ex_trap_c + 0x5c }
2 Target : <0xffa0057c> { _ex_trap_c + 0x0 }
Source : <0xffa007d4> { _trap + 0x28 }
3 Target : <0xffa007ac> { _trap + 0x0 }
Source : <0x0332eb82> [ /lib/libuClibc-0.9.29.so + 0x2eb82 ]
4 Target : <0x0332eb4a> [ /lib/libuClibc-0.9.29.so + 0x2eb4a ]
Source : <0x0332eb10> [ /lib/libuClibc-0.9.29.so + 0x2eb10 ]
5 Target : <0x0332eb08> [ /lib/libuClibc-0.9.29.so + 0x2eb08 ]
Source : <0x0332eb1c> [ /lib/libuClibc-0.9.29.so + 0x2eb1c ]
6 Target : <0x0332eb08> [ /lib/libuClibc-0.9.29.so + 0x2eb08 ]
I was also using gdb to debug the same. Following is the backtrace for which the crash
has occured.
#0 __heap_free (heap=0x2cd1510, mem=0x14a8, size=175399694) at libc/stdlib/malloc/heap.h:167
#1 0x0332e760 in free (mem=<value optimized out>) at libc/stdlib/malloc/free.c:45
#2 0x0314a540 in FreeStringArrayVariable ()
From this backtrace it appears that heap area of my application is getting corrupted.( Invalid, size=175399694)
Our application involves some third party libraries. I have walked through the entire code
flow for which we have control over and it looks fine.
We also verified with some static analyzer tools and there is no memory leaks identified with them too.
we are also using the buildflag "-mstack-check-l1" to compile our application to identify any StackOverFlow.
I am finding it difficult to detect the original cause of corruption as the buffer that lead to corruption could have been done already and the crash is triggered only when we access it.
I have planned to do the below
I would like to link the library statically so that i can keep some conditional break points at the size parameter and could do some backtrace. The toolchain which we were using is 2008R1.5 This is to identify the data area in code which is getting corrupted/overwritten.
Please let me know how to do static linking of the uclibc libraries ?
Also valuable comments on how to debug the above problem would be of great help to me.
Thanks in advance for your help.
Regards,
Jayaprabhu
QuoteReplyEditDelete
2009-06-17 11:52:11 Re: Reg: Static Linking of uclibc lib to track heap area corruption
Robin Getz (UNITED STATES)
Message: 75904
Jayaprabhu:
Why not read the documentation, and do what it says?
https://docs.blackfin.uclinux.org/doku.php?id=uclinux-dist:debugging_applications#stack_checking
https://docs.blackfin.uclinux.org/doku.php?id=uclinux-dist:debugging_applications#mudflap
-Robin
QuoteReplyEditDelete
2009-06-17 12:04:29 Re: Reg: Static Linking of uclibc lib to track heap area corruption
Mike Frysinger (UNITED STATES)
Message: 75908
use the -Bdynamic and -Bstatic linker flags can be used to selectively link libraries
gcc .... -Wl,-Bstatic,-lfoo,-Bdynamic ...
QuoteReplyEditDelete
2009-06-17 12:16:22 Re: Reg: Static Linking of uclibc lib to track heap area corruption
Jayaprabhu Nadarajan (INDIA)
Message: 75912
Robin:
I am already using the copiler flag "-mstack-check-l1" for building my application
Thanks,
Jayaprabhu
QuoteReplyEditDelete
2009-06-17 18:28:10 Re: Reg: Static Linking of uclibc lib to track heap area corruption
Robin Getz (UNITED STATES)
Message: 75921
Jayaprabhu:
And do you get stack overflow messages?
Did you try mudflap?
QuoteReplyEditDelete
2009-06-18 00:33:15 Re: Reg: Static Linking of uclibc lib to track heap area corruption
Jayaprabhu Nadarajan (INDIA)
Message: 75928
Robin,
I didnot get any stack overflow messages with "-mstack-check-l1". Havnt tried mudflap yet. Will try that too.
Thanks,
Jayaprabhu