2009-04-15 08:14:18 kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 72687
Hello,
When I power-cycle, the booting hangs after the message "Starting Kernel at = xxxx" (the EZice shows it's hanging in a loop MNOP || [P0++] = R3 || R3 = [I1++])
However, if I use the reset button, booting is running fine.
If I disable the Memory Optimization "Locate kernel schedule function in L1 Memory", the problem disappears (so it's working fine again). Any idea what's wrong?
I am using the blackfin 537 stamp board.
URL: svn://firewall-sources.blackfin.uclinux.org:80/svn/linux-kernel/trunk
Repository Root: svn://firewall-sources.blackfin.uclinux.org:80/svn/linux-kernel
Repository UUID: 526b6c2d-f592-4532-a319-5dd88ccb003d
Revision: 6268
URL: svn://firewall-sources.blackfin.uclinux.org:80/svn/uclinux-dist/trunk
Repository Root: svn://firewall-sources.blackfin.uclinux.org:80/svn/uclinux-dist
Repository UUID: 3e2a5524-39e8-4cb9-84a9-c49da69cba3a
Revision: 7972
URL: svn://firewall-sources.blackfin.uclinux.org:80/svn/u-boot/trunk/u-boot-2009.03
Repository Root: svn://firewall-sources.blackfin.uclinux.org:80/svn/u-boot
Repository UUID: e8ad1c53-384c-47db-b5ce-66ae1f620dea
Revision: 1842
Best regards,
Filip
TranslateQuoteReplyEditDelete
2009-04-16 00:13:13 Re: kernel boot fails after power-cycle, but reset-button works fine
Yi Li (CHINA)
Message: 72720
Filip,
How can I reproduce your issue?
You can do "make bugreport" in uclinux-dist and post the bugreport.tar.gz here. Also you can try to debug as:
http://docs.blackfin.uclinux.org/doku.php?id=linux-kernel:debug:early_printk
http://docs.blackfin.uclinux.org/doku.php?id=linux-kernel:debug:post_mortem
Since you have an ICE, you can also debug using GDB.
-Yi
QuoteReplyEditDelete
2009-04-17 15:47:20 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 72802
Filip:
Just so I understand - the first time you boot - it doesn't work, but the 2nd time you boot - it works?
-Robin
QuoteReplyEditDelete
2009-04-20 07:09:19 Re: kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 72862
Hello Li and Robin,
Thank you for your interest in my problem (sorry for my late reaction).
So my system hangs after a power-cycle, but not after a reset with the reset button.
I was working with a version of the kernel with some modifications (running the kernel from timer0 instead of the core timer and some more modifs). That worked fine till my last update from subversion. I now see that when I am working with an unmodified kernel, it always boots fine. So I'll try to debug it further myself.
Thanks,
Filip
TranslateQuoteReplyEditDelete
2009-04-21 05:46:42 Re: kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 72935
Hello,
Further investigation make me think that there is a risk for a race condition when the scheduler (as an optimization enabled by menuconfig) is moved to internal memory : this is the call tree
bfin_relocate_l1_mem -> dma_memcpy -> __dma_memcpy
this function contains :
bfin_write_MDMA_SO_CONFIG(xxx);
bfin_write_MDMA_DO_CONFIG(xxx);
spin_unlock_irq_restore(xxx);
spin_unlock_irq_restore will call preempt_schedule ... but this is the code that is being booted by DMA.
Please comment. (When using the reset button, the code is still in memory, but when power-cycling not all the code might yet been loaded).
For your infomation, I've put some more code in internal memory to have shorter interrupt-delays. Maybe this can be added to the kernel code (see attachment).
Best regards,
Filip
svndiff.txt
TranslateQuoteReplyEditDelete
2009-04-21 08:17:03 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 72978
Filip:
Thanks for tracking this down -- I will fix later today.
As for the patch - I have something along the same lines - I will review and send out later...
-Robin
QuoteReplyEditDelete
2009-04-21 14:30:28 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 73016
Filip:
Can you try this patch out, and see if it helps?
-Robin
setup.patch
QuoteReplyEditDelete
2009-04-21 14:40:16 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 73017
I forgot to say - I saw about a 10% speed up (in the time it takes to copy things from external memory to L1) with the patch. 394479 cycles (or 788us) as is, and with the patch - 355534 cycles (or 711 useconds) -- I don't think many are going to notice the 77us boot difference - but every little bit helps...
-Robin
QuoteReplyEditDelete
2009-04-22 03:50:43 Re: kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 73041
Hello Robin,
Thank you for your patch-file. There is still a race condition if memcpy is also in L1 (CONFIG_MEMCPY_L1 enabled) . But to solve this problem, only a minor change to your patch is required: either wait for the DMA to finish before the memcpy (this way the boottime will not get shorter than than before) or make a local implementation of memcpy.
Best regards,
Filip
TranslateQuoteReplyEditDelete
2009-04-22 09:39:33 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 73062
Filip:
Yeah, you are right. Let me see what I can do...
-Robin
QuoteReplyEditDelete
2009-04-22 10:40:10 Re: kernel boot fails after power-cycle, but reset-button works fine
Robin Getz (UNITED STATES)
Message: 73064
Filip:
Have a go with this.
Thanks
setup.patch
QuoteReplyEditDelete
2009-04-23 03:16:56 Re: kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 73086
Thanks Robin, this works fine.
Can you also enable the possibility to move all parts of an ISR to L1 (to get the fastest interrupt responce time possible) in the code of the kernel? I recently uploaded the patch in this thread.
Best regards,
Filip.
TranslateQuoteReplyEditDelete
2009-04-23 10:34:06 Re: kernel boot fails after power-cycle, but reset-button works fine
Mike Frysinger (UNITED STATES)
Message: 73107
we are avoiding touching common code just to add our attributes. there was discussion on our devel mailing list for a different (common) solution.
QuoteReplyEditDelete
2009-04-24 03:07:10 Re: kernel boot fails after power-cycle, but reset-button works fine
Filip Van Rillaer (BELGIUM)
Message: 73118
Ok, thank you for your answer Mike.
Best regards,
Filip