2009-04-21 14:00:09 BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73014
Hi PTG
Is there any known stability issues on the 2008r1 release for BF537 when using the bfin_mac.c driver and UART in PIO mode? We are seeing a crash happening with this combination.
-Hans
QuoteReplyEditDelete
2009-04-21 15:03:17 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73021
Hans:
Nope - none that are known - do you have any steps to reproduce.
-Robin
QuoteReplyEditDelete
2009-04-21 16:02:46 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73024
We are running the kernel on our BF537 based VoIP HW.
After the kernel boots up, we run a simple socket application which opens a UDP socket and does a blocking read on the socket repeatedly. From a Windows PC server, we send around 10 Mbps data to that same socket.
It is hanging in the function netif_receive_skb(). I think some list pointers are getting corrupted.
This same setup works fine if the UART is in DMA mode, we have discovered.
QuoteReplyEditDelete
2009-04-21 16:41:46 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73026
Hans:
There have been a few fixes in both the ethernet and UART code - are you using the branch or the tar ball?
-Robin
QuoteReplyEditDelete
2009-04-21 16:58:19 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73028
I guess it is a bit older version: 2008r1.
I will have to check with the team how we downloaded the sources.
QuoteReplyEditDelete
2009-04-21 17:10:09 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73029
Hans:
There were quite a few updates after the tar balls were made - that could explain things.
-Robin
QuoteReplyEditDelete
2009-04-22 09:14:56 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73060
Thanks, Robin
We will try to see if any of the fixes solve the problem.
-Hans
QuoteReplyEditDelete
2009-04-23 16:07:36 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73111
We changed the UART mode to DMA. Here the crash is happening less often. I tried adding the adding the latest fixes from the 2008r1 SVN branch. But this did not help.
I have arrived at the following trace info: The crash is happening in the file /net/core/dev.c in the function netif_receive_skb() in the loop:
list_or_each_entry_rcu(ptype, &ptype_base[ntohs(type)&15], list) {
if (ptype->type == type &&
(!ptype->dev || ptype->dev == skb->dev)) {
if (pt_prev)
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = ptype;
}
}
I have traced that the crash is happening due to one of the ptype list entry member list.next pointer is becoming 0, which is not good in this type of doubly linked list.
I am also seeing that this is consistently happening after we output something on the console using printf.
I see there are many changes in the bfin_mac.c driver from version 2008r1 to the latest 2009 trunk version. Some of these are DMA related. Would it make sense for me to merge some of these fixes back into the 2008r1 driver? If so, which of these fixes sound like they could solve the problem I am seeing?
Unfortunately we are not well positioned to just move to the latest kernel version completely.
-Hans
QuoteReplyEditDelete
2009-04-23 17:14:07 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73112
Hans:
What would help the most is a test case - We do lots of networking/UART tests, so - while there could always be some issue that we haven't seen - I'm pretty sure it works.
What you have so far - is a report saying that memory is getting clobber by something - in noMMU - that could be anything - including your userspace application....
-Robin
QuoteReplyEditDelete
2009-04-24 10:10:55 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73160
OK, Robin.
How do I need to provide it to you? Does it need to be on EZ-kit or something like that? Is it OK if it is 2008r1 version?
When would you be able to schedule debugging, if we can reproduce this in your environment.
-Hans
QuoteReplyEditDelete
2009-04-24 10:21:43 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73161
Hans:
Something on an EZkit/stamp is best - that way we don't need to shuffle hardware around the planet.
If you have a test case that fails on 2008R1.5 - the first thing we will do is try it out on trunk - and see if the failure still exists there. If it does - we will fix it in both places.
As soon as you have a test case - we can start looking at it.
-Robin
QuoteReplyEditDelete
2009-04-24 11:03:00 Re: BF537 MAC crash with UART in PIO mode
Hans Waldmann (UNITED STATES)
Message: 73165
Robin,
Our customers are still using 2008r1 release. We need to support them on that release for some time longer. It it possible you can support us on that release also?
Would you happen to have a spare EZ-kit board we could borrow for this exercise? If not, then I will go ahead and order one.
-Hans
QuoteReplyEditDelete
2009-04-24 11:15:54 Re: BF537 MAC crash with UART in PIO mode
Robin Getz (UNITED STATES)
Message: 73167
Hans:
No -- I don't have any spare EZkits. Better to use a STAMP (which I don't have spares of either) -- as that is what we use for normal development.
As for what _we_ support - we support trunk (head) and the latest branch (head) - at this current moment - that is 2008R1 (the tarball was called 2008R1.5).
blackfin.uclinux.org/gf/project/linux-kernel/scmsvn/?action=browse&path=/branches/2008R1/
which was last modified 29 hours ago -- (like I said, we do support the latest release). If a fix was made in the branch that you aren't using - then there isn't anything we are going to do about it.
-Robin