2009-05-19 07:55:10     Networking issues after real time task

Document created by Aaronwu Employee on Aug 15, 2013
Version 1Show Document
  • View in full screen mode

2009-05-19 07:55:10     Networking issues after real time task

Michael Löffler (GERMANY)

Message: 74255   

 

Hello!

 

I have an bf537 stamp board running with uclinux and xenomai. For some applications we have real time tasks, which last up to five seconds. Everything seems to be fine with this, except the network stack. I don't even have a clue where to look for the error, as the processor load seems to be fine, but the board is merely not responsible via network. For example, after one of these tasks, a simple ping to the blackfin looks like this:

 

PING bfin (__ip__) 56(84) bytes of data.

64 bytes from bfin (__ip__): icmp_seq=11 ttl=64 time=0.370 ms

64 bytes from bfin (__ip__): icmp_seq=5 ttl=64 time=6000 ms

64 bytes from bfin (__ip__): icmp_seq=6 ttl=64 time=5000 ms

64 bytes from bfin (__ip__): icmp_seq=7 ttl=64 time=4000 ms

64 bytes from bfin (__ip__): icmp_seq=8 ttl=64 time=3000 ms

64 bytes from bfin (__ip__): icmp_seq=9 ttl=64 time=2000 ms

64 bytes from bfin (__ip__): icmp_seq=10 ttl=64 time=1000 ms

64 bytes from bfin (__ip__): icmp_seq=17 ttl=64 time=0.214 ms

64 bytes from bfin (__ip__): icmp_seq=18 ttl=64 time=0.335 ms

64 bytes from bfin (__ip__): icmp_seq=12 ttl=64 time=6000 ms

64 bytes from bfin (__ip__): icmp_seq=13 ttl=64 time=5000 ms

64 bytes from bfin (__ip__): icmp_seq=14 ttl=64 time=4000 ms

64 bytes from bfin (__ip__): icmp_seq=15 ttl=64 time=3000 ms

64 bytes from bfin (__ip__): icmp_seq=16 ttl=64 time=2000 ms

64 bytes from bfin (__ip__): icmp_seq=26 ttl=64 time=0.221 ms

64 bytes from bfin (__ip__): icmp_seq=10 ttl=64 time=16360 ms (DUP!)

64 bytes from bfin (__ip__): icmp_seq=20 ttl=64 time=6360 ms

64 bytes from bfin (__ip__): icmp_seq=21 ttl=64 time=5360 ms

64 bytes from bfin (__ip__): icmp_seq=22 ttl=64 time=4360 ms

64 bytes from bfin (__ip__): icmp_seq=23 ttl=64 time=3360 ms

64 bytes from bfin (__ip__): icmp_seq=24 ttl=64 time=2361 ms

64 bytes from bfin (__ip__): icmp_seq=25 ttl=64 time=1361 ms

--- bfin ping statistics ---

29 packets transmitted, 21 received, +1 duplicates, +3 errors, 27% packet loss, time 28020ms

rtt min/avg/max/mdev = 0.214/3660.634/16360.671/3443.366 ms, pipe 17

 

 

top:

 

   18 root     SW<        0     2 99.8  0.0 IRQ 26

   41 root     S        556     1  0.0  2.5 sh

   43 root     S        464     1  0.0  2.1 klogd

   93 root     R        464    41  0.0  2.1 top

   42 root     S        456     1  0.0  2.1 syslogd

    1 root     S        104     0  0.0  0.4 init

   39 root     S         40     1  0.0  0.1 inetd

    2 root     SW<        0     0  0.0  0.0 kthreadd

    4 root     SW<        0     2  0.0  0.0 events/0

    6 root     SW<        0     2  0.0  0.0 kblockd/0

    3 root     SWN        0     2  0.0  0.0 ksoftirqd/0

   12 root     SW<        0     2  0.0  0.0 mtdblockd

   13 root     SW<        0     2  0.0  0.0 bfin-spi.0

   14 root     SW<        0     2  0.0  0.0 IRQ 9 (demux)

   15 root     SW<        0     2  0.0  0.0 IRQ 18

   16 root     SW<        0     2  0.0  0.0 IRQ 19

   17 root     SW<        0     2  0.0  0.0 IRQ 24

    5 root     SW<        0     2  0.0  0.0 khelper

   19 root     SW<        0     2  0.0  0.0 IRQ 34 (demux)

    7 root     SW<        0     2  0.0  0.0 gatekeeper/0

    8 root     SW         0     2  0.0  0.0 pdflush

 

 

Any ideas would be really appreciated!

TranslateQuoteReplyEditDelete

 

 

2009-05-19 08:00:04     Re: Networking issues after real time task

Michael Löffler (GERMANY)

Message: 74256   

 

I forgot:

 

version:

 

kernel:    Linux release 2.6.22.19-ADI-2008R1.5-svn, build #50 Mon Mar 30 16:489

toolchain: bfin-uclinux-gcc release gcc-Version 4.1.2 (ADI svn)               

user-dist: release 2008R1.5, build #16 Mo 30. Mär 16:46:53 CEST 2009 

TranslateQuoteReplyEditDelete

 

 

2009-05-19 08:16:22     Re: Networking issues after real time task

Mike Frysinger (UNITED STATES)

Message: 74259   

 

not letting Linux run for 5 seconds is pretty bad.  does the system fail if you set the timeout to something lower and more reasonable ?

 

do you have the Blackfin MAC driver setup for using L1 data to receive packets ?

QuoteReplyEditDelete

 

 

2009-05-19 13:43:08     Re: Networking issues after real time task

Michael Löffler (GERMANY)

Message: 74276   

 

I know that it's not a perfect solution to stop linux completely, but we have to collect and process sensor data with ~100kHz and a fixed delay, so if I allow linux to run, the delay is too high. The networking keeps functional, as long as the process lasts not more than about one second. More than this, and the networking will be corrupted.

 

While this measuring process, it's not even necessary that the driver receives any packages, it just shouldn't get corrupted. Disabling the network completely would be a possible workaround, but I'd prefer a solution closer to the root of the problem and also understand, what's going wrong there.

 

For the MAC driver setup: I haven't changed anything about it, it's the original configuration of the distribution.

TranslateQuoteReplyEditDelete

 

 

2009-05-19 18:29:04     Re: Networking issues after real time task

Mike Frysinger (UNITED STATES)

Message: 74292   

 

i'm not sure Linux is able to be suspended for that long.  saying you only noticed a problem with the networking and therefore it's (1) a networking problem and (2) everything else works fine is not a bulletproof argument.  istr the issue being larger than that (the internal jiffies werent being updated properly).

 

please open a tracker item with all your configuration info so someone can take a look.

QuoteReplyEditDelete

 

 

2009-05-19 21:55:52     Re: Networking issues after real time task

Yi Li (CHINA)

Message: 74295   

 

I have two suggestions:

 

1. You may post the question on xenomai mailing list.

 

2. You may also try xenomai/adeos on SVN trunk.

QuoteReplyEditDelete

 

 

2009-05-19 23:40:26     Re: Networking issues after real time task

Sonic Zhang (CHINA)

Message: 74296   

 

I guess you should set the ethernet rx/tx dma buffers to the lagest value. Stopping kernel for such a long time may overflow the ethernet rx/tx DMA buffers.

QuoteReplyEditDelete

 

 

2009-05-20 17:55:26     Re: Networking issues after real time task

Michael Löffler (GERMANY)

Message: 74343   

 

Ok, thanks for the suggestions. I'll have a look at the rx/tx buffer.

TranslateQuoteReplyEditDelete

Attachments

    Outcomes