2009-11-11 19:07:16     gpio outputs causing missed edge-trigger interrupts?

Document created by Aaronwu Employee on Aug 19, 2013
Version 1Show Document
  • View in full screen mode

2009-11-11 19:07:16     gpio outputs causing missed edge-trigger interrupts?

Ron Weiland (UNITED STATES)

Message: 82278   

 

On my custom BF533-based board, I have a driver that operates on a 500us timer interrupt and uses a bit-banged serial interface to talk to a Xilinx CPLD.  Using 2008 release, everything is fine.  Using 2009 release, after just a minute or two I lose my Ethernet and eventually get the following error:

 

------------[ cut here ]------------

WARNING: at net/sched/sch_generic.c:226 _dev_watchdog+0x204/0x20c()

NETDEV WATCHDOG: eth0 (dm9000): transmit timed out

Modules linked in:

Hardware Trace:

   0 Target : <0x00004b44> { _dump_stack + 0x0 }

     Source : <0x0000fe40> { _warn_slowpath + 0x58 } CALL pcrel

   1 Target : <0x0000fe40> { _warn_slowpath + 0x58 }

     Source : <0x00028c2e> { _print_modules + 0x7a } RTS

   2 Target : <0x00028c28> { _print_modules + 0x74 }

     Source : <0x00010492> { _printk + 0x16 } RTS

   3 Target : <0x0001048e> { _printk + 0x12 }

     Source : <0x00010d0a> { _vprintk + 0x132 } RTS

   4 Target : <0x00010ce8> { _vprintk + 0x110 }

     Source : <0x00010e2e> { _vprintk + 0x256 } JUMP.S

   5 Target : <0x00010e2e> { _vprintk + 0x256 }

     Source : <0x00010306> { _wake_up_klogd + 0x1a } RTS

   6 Target : <0x00010306> { _wake_up_klogd + 0x1a }

     Source : <0x000102f8> { _wake_up_klogd + 0xc } IF !CC JUMP

   7 Target : <0x000102ec> { _wake_up_klogd + 0x0 }

     Source : <0x0001071a> { _release_console_sem + 0x1ae } JUMP.L

   8 Target : <0x00010712> { _release_console_sem + 0x1a6 }

     Source : <0x000106f6> { _release_console_sem + 0x18a } IF !CC JUMP

   9 Target : <0x000106ea> { _release_console_sem + 0x17e }

     Source : <0x00023372> { _up + 0x3e } RTS

  10 Target : <0x0002336c> { _up + 0x38 }

     Source : <0x0002335e> { _up + 0x2a } IF !CC JUMP

  11 Target : <0x00023334> { _up + 0x0 }

     Source : <0x000106e6> { _release_console_sem + 0x17a } CALL pcrel

  12 Target : <0x000106d2> { _release_console_sem + 0x166 }

     Source : <0x000105ba> { _release_console_sem + 0x4e } IF !CC JUMP

  13 Target : <0x0001058e> { _release_console_sem + 0x22 }

     Source : <0x000106b4> { _release_console_sem + 0x148 } IF CC JUMP

  14 Target : <0x000106ac> { _release_console_sem + 0x140 }

     Source : <0x0001025e> { __call_console_drivers + 0x7a } RTS

  15 Target : <0x00010258> { __call_console_drivers + 0x74 }

     Source : <0x0001021a> { __call_console_drivers + 0x36 } IF !CC JUMP

Stack info:

SP: [0x0022fd04] <0x0022fd04> /* kernel dynamic memory */

FP: (0x0022fdf0)

Memory from 0x0022fd00 to 00230000

0022fd00: 001c8ed0 [0000fe40]<0000fe44> 0022fd24  001a31f8  001c31c8  001a31f8  000000e2

0022fd20: 0022fd24  7665645f  7461775f  6f646863  78302b67  2f343032  30327830  00000063

0022fd40: 00000004  00000100 <0002d854> 001cea90  00000006  00227058  00000006  00d31140

0022fd60: ffa0032e  001c68c0  00000005  00227058  00000006  00000000  ffa0032e  001c68c0

0022fd80: 0029fca0  0022fd84  0022fd84  ffa00bc2  ffc00014  00226000  00000005  00230000

0022fda0: 0098fa82  000a4530  ffa0018a  00008050  00000000  00000000  00230000  0098fa82

0022fdc0: ffa0018a <0006e046> 00000006  02002022  009a162f  009c218d  009a162e  009c2180

0022fde0: 00000000  00000000  00000000  00000000 (00000000)<00101548> 0000003f  ffffffff

0022fe00: ffffffc0  0022fe28 <0010e4a8> 001cdc34  0000003f  ffffffff  ffffffc0  07bdaaa4

0022fe20: 00000000  001c46a0  00b2b000  0022fe30  30396d64 <00003030><0000ddde><07b8240c>

0022fe40: 0022fe60  00000000  00000001  00000006  07bdaaa4  07b823e0 <0001788a> 0029fd1c

0022fe60: 0022fe60  0022fe60  0022fea4 <00017430><000174cc> 001c32e0  0022fe9c  0010e2a4

0022fe80: 0022e000  00000100  001cc038  00000000  00000004  0029fca0 <0003f818> 001c4c2c

0022fea0: 001c4c2c  00000001 <0001438e> 001c31c8  0022e000  001c315c  00000001  00000004

0022fec0: 00000100 <0002d854> 001cea90  00000006  002b91b0  00000006  0029fd1c  ffa0032e

0022fee0: 001c68c0  00227004  002b91b0  00000006  00000000  00000001  07bdaaa4  ffa00880

0022ff00: 0003fb08  00000000  ffa00bc2  ffc00014  00000005  002bd238  00000000  00000004

0022ff20: 0098fa82  009c218c  00000050  00000000  00000000  00230000  0098fa82  009c218c

0022ff40:<009a4d52> 00000006  02002022  009a162f  009c218d  009a162e  009c2180  00000000

0022ff60: 00000004  00000000  00000000  00000000  00000000  00000000  00000000  00000000

0022ff80: 00000000  00000000  00000000  00000000  00000000  00000000  00000000  00000000

0022ffa0: 00000000  00000000  00000000  009bce80  0029fc95  0029fd10  0029fd1c  00227004

0022ffc0: 00227004  002b91b0  00000005  009c2178  00000020  00000005  002bd238  00000001

0022ffe0: 07bdaaa4  00000000  ffffffff  00000001  bfffffff  bfffffff  00000020  00000006

Return addresses in stack:

    address : <0x0000fe40> { _warn_slowpath + 0x58 }

    address : <0x0000fe44> { _warn_slowpath + 0x5c }

    address : <0x0002d854> { _handle_simple_irq + 0x74 }

    address : <0x0006e046> { _sysfs_write_file + 0x36 }

   frame  1 : <0x00101548> { _netdev_drivername + 0x34 }

    address : <0x0010e4a8> { _dev_watchdog + 0x204 }

    address : <0x00003030> { _do_signal + 0x434 }

    address : <0x0000ddde> { _task_tick_fair + 0x1e }

    address : <0x07b8240c> /* kernel dynamic memory */

    address : <0x0001788a> { _update_process_times + 0x1e }

    address : <0x00017430> { _run_timer_softirq + 0x14 }

    address : <0x000174cc> { _run_timer_softirq + 0xb0 }

    address : <0x0003f818> { _vfs_write + 0xac }

    address : <0x0001438e> { ___do_softirq + 0x5a }

    address : <0x0002d854> { _handle_simple_irq + 0x74 }

    address : <0x009a4d52> [ /lib/libc.so.0 + 0x24d52 ]

---[ end trace 487c1a7d0e18d968 ]---

 

 

Here is the /proc/interrupts after boot:

 

  6:       3572      CORE  Blackfin CoreTimer

14:          1      INTN  rtc-bfin

21:          0      INTN  BFIN_UART_RX

22:        895      INTN  BFIN_UART_TX

44:          2      GPIO  isp1362-hcd:usb1

48:          3      GPIO  eth0

NMI:          0      CORE  Non Maskable Interrupt

Err:          0

 

FIO_MASKA_D is 0x8800, FIO_MASKB_D is 0x0000

 

 

To eliminat e other possibiliites, I wrote a short test program that uses the gpio_sysfs interface to just toggle a single GPIO output line as fast as possible.  No matter which flag I toggle, I get the same problem. Slowing the application with nanosleep just makes it take longer for the problem to occur.

 

If I set the Ethernet to level-triggered instead of edge-triggered in my board resource file, I never lose the Ethernet but I don't know if I'm simply masking the real issue.

 

Any insight you can give would be much appreciated.

 

Thanks!

 

Ron

QuoteReplyEditDelete

 

 

2009-11-12 15:20:04     gpio outputs causing missed edge-trigger interrupts?

Michael Hennerich (GERMANY)

Message: 82329    Ron,

 

Using /sys/kernel/debug/blackfin

can you cat and post all PF/GPIO regs before and after the issue happens?

 

-Michael

QuoteReplyEditDelete

 

 

2009-11-12 21:07:37     gpio outputs causing missed edge-trigger interrupts?

Ron Weiland (UNITED STATES)

Message: 82335    Thanks for responding, Michael

 

Here are the GPIO registers before and after the "issue".

I don't really see anything revealing, maybe you do. Let me know if I

am missing any relevant registers you would like to see.

 

The "before" is during the test but before lockup.

"After" was during the test, after the system stopped responding to

Ethernet interrupts.

The line that was being toggled was PF8.

 

Before After

 

FIO_BOTH = 0x0000 0x0000

FIO_DIR = 0x2104 0x2104

FIO_EDGE = 0x8000 0x8000

FIO_FLAG_D = 0x0504 0x0404

FIO_FLAG_C = 0x0404 0x0404

FIO_FLAG_S = 0x0504 0x0404

FIO_FLAG_T = 0x0404 0x0404

FIO_INEN = 0x8c00 0x8c00

FIO_MASKA_D = 0x8800 0x8800

FIO_MASKA_C = 0x8800 0x8800

FIO_MASKA_S = 0x8800 0x8800

FIO_MASKA_T = 0x8800 0x8800

FIO_MASKB_C = 0x0000 0x0000

FIO_MASKB_D = 0x0000 0x0000

FIO_MASKB_S = 0x0000 0x0000

FIO_MASKB_T = 0x0000 0x0000

FIO_POLAR = 0x0800 0x0800

 

ILAT = 0x00000000 0x00000000

IMASK = 0x0000ffff 0x0000ffff

IPEND = 0x00008000 0x00008000

IPRIO = 0x00000000 0x00000000

 

Ron

QuoteReplyEditDelete

 

 

2009-11-13 05:26:59     gpio outputs causing missed edge-trigger interrupts?

Michael Hennerich (GERMANY)

Message: 82354    Hi Ron,

 

Are you reading IRQ48 aka. PF15 while eth0 is active?

Are you reading any other GPIO?

 

So you just toggle PF8...

set_gpio_data didn't change...

This is strange - I'm not aware of significant differences between 2008R1 and 2009R1, in case you just toggle a different gpio.

 

Just to see what happens -

In case our Ethernet is locked up - clear FIO_EDGE = 0x0000

And then read FIO_FLAG_D -

 

I bet PF15 is active - and for some reason we lost the transitioning edge.

This will explain why you don't see any further interrupts.

 

Which BF533 silicon version are you using - and how did you configure your kernel?

 

Please attach your kernel .config and your full kernel startup messages.

 

This sounds like ANOMALY

05000277 - Writes to an I/O Data Register One SCLK Cycle after an Edge Is Detected May Clear Interrupt:

 

But I wonder why you didn't see it with 2008R1.

 

In case you use Silicon Rev. <= 0.5

Consider using level sensitive interrupts if possible.

 

-Michael

QuoteReplyEditDelete

 

 

2009-11-16 16:38:01     Re: gpio outputs causing missed edge-trigger interrupts?

Ron Weiland (UNITED STATES)

Message: 82418   

 

I'm not doing any reading of IRQ48 except what the kernel and driver are doing.

The only application running at the time is the test program that toggles the PF line.

We have about 100 systems in the field running 2008, and I don't know of any cases where the Ethernet has locked up.

 

In spite of that, the problem does sound a lot like anomaly 05000277 and the silicon rev is 0.5.  I wish I had a higher rev to test with, but I don't yet.  I know it isn't the board itself, because I'm doing the 2008 vs. 2009 test on the same board.  The 2008 is programmed in NAND, the 2009 I load with tftp from u-boot.

 

I've included the u-boot / kernel startup messages and the linux .config.   I think for now, I'll just go ahead and set to level-triggered interrupts in the board resource file.

 

Thank you so much for all your help, Michael!

 

Ron Weiland

 

config

bootup.txt

Attachments

Outcomes