2009-07-17 16:43:20     Trouble mounting filesystems during startup

Document created by Aaronwu Employee on Aug 16, 2013
Version 1Show Document
  • View in full screen mode

2009-07-17 16:43:20     Trouble mounting filesystems during startup

Steve Strobel (UNITED STATES)

Message: 77584   

 

I recently started having trouble mounting filesystems during startup, getting these messages:

 

mount: mounting proc on /proc failed

mount: mounting ramfs on /var failed

mount: mounting sysfs on /sys failed

mount: mounting devpts on /dev/pts failed

 

After it boots, I can manually run /etc/rc and it does those mounts without any problems.  I tried adding "sleep 5" to the rc file before doing that mounts;  that didn't help.

 

I can revert to an older build (same version of uClinux but various other differences) and get it to work, but I haven't yet been able to figure out what changed and broke it.  I can build the current version for other targets and it works fine.  All of the targets I build for use the same /etc/rc file, which begins with the lines:

 

#! /bin/sh

mount -t proc proc /proc

mount -t ramfs ramfs /var

mount -t sysfs sysfs /sys

mount -t devpts devpts /dev/pts

 

A complete startup log is attached below.  I have compared it with the log from the earlier (working) version and didn't find any significant differences.

 

Thanks for suggestions about what to look for,

 

Steve

 

Linux version 2.6.22.19-ADI-2008R1.5-svn542 (stevestrobel@dev1-ubuntu) (gcc version 4.1.2 (ADI svn)) #51 Thu Jul 16 13:2

4:00 MDT 2009

Hardware Trace Active and Enabled

Memory size limited to 34MB due to 05000263

Blackfin support (C) 2004-2007 Analog Devices, Inc.

Compiled for ADSP-BF537 Rev 0.2

Warning: Compiled for Rev 2, but running on Rev 3

Blackfin Linux support by   blackfin.uclinux.org/

Processor Speed: 500 MHz core clock and 100 MHz System Clock

Board Memory: 64MB

Kernel Managed Memory: 64MB

Memory map:

  text      = 0x00001000-0x00121350

  rodata    = 0x00122000-0x0016f018

  data      = 0x00170000-0x00182000

    stack   = 0x00170000-0x00172000

  init      = 0x00182000-0x00193000

  bss       = 0x00193000-0x001a3830

  available = 0x001a3830-0x02200000

  rootfs    = 0x02200000-0x03f00000

  DMA Zone  = 0x03f00000-0x04000000

NOMPU: setting up cplb tables for global access

Instruction Cache Enabled

Data Cache Enabled (write-back)

Built 1 zonelists.  Total pages: 8636

Kernel command line: root=/dev/mtdblock0 rw console=ttyBF0,115200

Configuring Blackfin Priority Driven Interrupts

PID hash table entries: 256 (order: 8, 1024 bytes)

Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)

Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)

Memory available: 32560k/65536k RAM, (68k init code, 1152k kernel code, 454k data, 1024k dma, 30280k reserved)

Blackfin Scratchpad data SRAM: 4 KB

Blackfin Data A SRAM: 16 KB (15 KB free)

Blackfin Data B SRAM: 16 KB (16 KB free)

Blackfin Instruction SRAM: 48 KB (39 KB free)

Mount-cache hash table entries: 512

NET: Registered protocol family 16

Blackfin GPIO Controller

Blackfin DMA Controller

msat_ip_board_init(): registering device resources

msat_ip_board_init(): registering Blackfin SPI resources

msat_ip_board_init(): finished registering Blackfin SPI resources

Generic PHY: Registered new driver

NET: Registered protocol family 2

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)

TCP established hash table entries: 2048 (order: 2, 16384 bytes)

TCP bind hash table entries: 2048 (order: 1, 8192 bytes)

TCP: Hash tables configured (established 2048 bind 2048)

TCP reno registered

JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.

io scheduler noop registered

io scheduler anticipatory registered (default)

io scheduler cfq registered

ISA-BlackFin-CAN CAN Driver 3.4.6_AD_BLACKFIN (c) Jul 16 2009

BlackFin port by H.J. Oertel (oe@port.de)

bfin-wdt: initialized: timeout=20 sec (nowayout=0)

MSAT-IP cors driver

MSAT-IP dtmf_strobe driver

RLC-DSP4 dtmf_data driver

MSAT-IP init_switch driver

MSAT-IP leds driver

MSAT-IP ptts driver

Serial: Blackfin serial driver

bfin-uart.1: ttyBF0 at MMIO 0xffc00400 (irq = 18) is a BFIN-UART

RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize

bfin_mac_mdio: probed

bfin_mac: attached PHY driver [Generic PHY] (mii_bus:phy_addr=0:01, irq=-1, mdc_clk=2500000Hz(mdc_div=19)@sclk=100MHz)

bfin_mac: Version 1.1, Blackfin BF53[67] BF527 on-chip Ethernet MAC driver

uclinux[mtd]: RAM probe address=0x2200000 size=0x1d00000

Creating 1 MTD partitions on "RAM":

0x00000000-0x01d00000 : "ROMfs"

m25p80 spi0.1: m25p128 (16384 Kbytes)

Creating 4 MTD partitions on "m25p80":

0x00000000-0x00040000 : "bootloader"

0x00040000-0x00080000 : "bootloader-environment"

0x00080000-0x00e00000 : "kernel"

0x00e00000-0x01000000 : "file system"

bfin-spi bfin-spi.0: Blackfin BF5xx on-chip SPI Contoller Driver, Version 1.0, regs_base@ffc00500, dma channel@7

bfin-spi-gpio-mstr: registering platform driver

bfin-spi-gpio-mstr: bfin_spi_gpio_probe()

bits_per_word not specified in spi_bitbang_setup.  Defaulting to 32.

bits_per_word not specified in spi_bitbang_setup.  Defaulting to 32.

Advanced Linux Sound Architecture Driver Version 1.0.12rc1 (Thu Jun 22 13:55:50 2006 UTC).

dma_alloc_init: dma_page @ 0x002ef000 - 256 pages at 0x03f00000

snd_ad1938_configure

ALSA device list:

  #0: ADI ad1938 at PF4 SPORT0 rx/tx dma 3/4 err irq 45

TCP cubic registered

NET: Registered protocol family 1

NET: Registered protocol family 17

VFS: Mounted root (ext2 filesystem).

Freeing unused kernel memory: 68k freed

mount: mounting proc on /proc failed

mount: mounting ramfs on /var failed

mount: mounting sysfs on /sys failed

mount: mounting devpts on /dev/pts failed

mkdir: cannot create directory '/var/log': File exists

mkdir: cannot create directory '/var/run': File exists

                           _____________________________________

        a8888b.           / Welcome to the uClinux distribution \

       d888888b.         /       _     _                         \

       8P"YP"Y88        /       | |   |_|            __  __ (TM)  |

       8|o||o|88  _____/        | |    _ ____  _   _ \ \/ /       |

       8'    .88       \        | |   | |  _ \| | | | \  /        |

       8`._.' Y8.       \       | |__ | | | | | |_| | /  \        |

      d/      `8b.       \      \____||_|_| |_|\____|/_/\_\       |

     dP   .    Y8b.       \   For embedded processors including   |

    d8:'  "  `::88b        \    the Analog Devices Blackfin      /

   d8"         'Y88b        \___________________________________/

  :8P    '      :888

   8a.   :     _a88P         For further information, check out:

._/"Yaa_:   .| 88P|            -   blackfin.uclinux.org/

\    YP"    `| 8P  `.          -   docs.blackfin.uclinux.org/

/     \.___.d|    .'           -   www.uclinux.org/

`--..__)8888P`._.'  jgs/a:f    -   www.analog.com/blackfin

 

Have a lot of fun...

mount: mounting /dev/mtdblock4 on /mnt/permanent failed

This is /mnt/permanent/scripts/init-passwd

Checking for password file

Setting up default password

Password file exists - using it

 

 

BusyBox v1.4.1 (2009-07-16 14:04:37 MDT) Built-in shell (msh)

Enter 'help' for a list of built-in commands.

 

root:~>

QuoteReplyEditDelete

 

 

2009-07-18 00:00:38     Re: Trouble mounting filesystems during startup

Mike Frysinger (UNITED STATES)

Message: 77593   

 

what if you run the first command through strace instead ?

 

strace mount -t proc proc /proc

 

then post the output you see from that

QuoteReplyEditDelete

 

 

2009-07-20 13:27:45     Re: Trouble mounting filesystems during startup

Steve Strobel (UNITED STATES)

Message: 77751   

 

Thanks for the reply, Mike.  The output from strace is included below, but I am not sure it is helpful, as the mount command works when run with strace.  The following mounts also succeed if the first one is run with strace.  In fact, just running "strace ls /" before the mounts makes all of them work.  I don't think the problem is timing related, as I previously inserted "sleep 5" at the very beginning of the rc file;  the beginning of it now looks like this:

 

#! /bin/sh

 

echo "#####################"

echo "### Customized rc ###"

echo "#####################"

 

echo "Sleeping..."

sleep 5

echo "...done sleeping."

 

# Try 1: This fails

# mount -t proc proc /proc

 

# Try 2: This works (and apparently causes the following mounts to work as well)

# strace mount -t proc proc /proc

 

# Try 3: Testing to see if calling strace on a different executable helps...yes

# strace ls /

# echo "---------"

# mount -t proc proc /proc

 

# Try 4: Exactly as above, except with the call to "strace ls /" removed - the mount fails

echo "---------"

mount -t proc proc /proc

 

echo "Sleeping..."

sleep 5

echo "...done sleeping."

 

mount -t ramfs ramfs /var

mount -t sysfs sysfs /sys

mount -t devpts devpts /dev/pts

 

Between each "Try X" above, I just ran "make romfs image", so the rc file should be the only thing that changed each time.  The output from Try 2 follows:

 

#####################

### Customized rc ###

#####################

Sleeping...

...done sleeping.

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x21fe000

stat("/etc/ld.so.cache", {st_mode=S_IFREG|0644, st_size=1183, ...}) = 0

open("/etc/ld.so.cache", O_RDONLY)      = 3

mmap2(NULL, 1183, PROT_READ, MAP_PRIVATE, 3, 0) = 0x4a2000

close(3)                                = 0

open("/lib/libgcc_s.so.1", O_RDONLY)    = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=47132, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a3000

read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0j\0\1\0\0\0(\37\0\000"..., 4096) = 4096

mmap2(NULL, 43520, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE|MAP_EXECUTABLE, 3, 0) = 0x4b0000

mmap2(NULL, 5148, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a4000

pread(3, "\201\t\1\350\0\0\1`\370c\1\6\245\5\20\0\1\350\0\0\10`\245"..., 5028, 40960) = 5028

close(3)                                = 0

munmap(0x4a3000, 4096)                  = 0

open("/lib/libc.so.0", O_RDONLY)        = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=236388, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a3000

read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0j\0\1\0\0\0\220\273"..., 4096) = 4096

mmap2(NULL, 229024, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE|MAP_EXECUTABLE, 3, 0) = 0x580000

mmap2(NULL, 18184, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a8000

pread(3, ": accept: %m\n\0\0\0socket: protocol"..., 9664, 225280) = 9664

close(3)                                = 0

munmap(0x4a3000, 4096)                  = 0

open("/lib/libc.so.0", O_RDONLY)        = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=236388, ...}) = 0

close(3)                                = 0

munmap(0x4a2000, 1183)                  = 0

stat("/lib/ld-uClibc.so.0", {st_mode=S_IFREG|0755, st_size=29824, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a2000

ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0x4dfa00) = -1 ENOTTY (Inappropriate ioctl for device)

ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B115200 opost isig icanon echo ...}) = 0

getuid32()                              = 0

getgid32()                              = 0

setgid32(0)                             = 0

setuid32(0)                             = 0

mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|0x4000000, 0, 0) = 0x4a6000

getcwd("", 4098)                        = 2

lstat("proc", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0

mount("proc", "/proc", "proc", MS_SILENT, "") = 0

_exit(0)                                = ?

Process 115 detached

Sleeping...

...done sleeping.

 

Thanks for any further suggestions.

 

Steve

QuoteReplyEditDelete

 

 

2009-07-20 14:18:05     Re: Trouble mounting filesystems during startup

Mike Frysinger (UNITED STATES)

Message: 77753   

 

wonder if it's a race condition ... try using '-o /log' rather than outputting to stdout

QuoteReplyEditDelete

 

 

2009-07-20 16:36:12     Re: Trouble mounting filesystems during startup

Steve Strobel (UNITED STATES)

Message: 77766   

 

I removed the five second delay at the beginning of /etc/rc and changed the first mount command to

 

strace -o /log mount -t proc proc /proc

 

The mount works fine.  The output in /log is included below, although since it doesn't fail, it probably isn't helpful.

 

Just to make sure nothing else changed, I removed the strace call and made sure that the mount still fails;  it does.

 

root:~> cat /log

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a2000

stat("/etc/ld.so.cache", {st_mode=S_IFREG|0644, st_size=1183, ...}) = 0

open("/etc/ld.so.cache", O_RDONLY)      = 3

mmap2(NULL, 1183, PROT_READ, MAP_PRIVATE, 3, 0) = 0x4a3000

close(3)                                = 0

open("/lib/libgcc_s.so.1", O_RDONLY)    = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=47132, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a4000

read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0j\0\1\0\0\0(\37\0\000"..., 4096) = 4096

mmap2(NULL, 43520, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE|MAP_EXECUTABLE, 3, 0) = 0x4b0000

mmap2(NULL, 5148, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a6000

pread(3, "\201\t\1\350\0\0\1`\370c\1\6\245\5\20\0\1\350\0\0\10`\245"..., 5028, 40960) = 5028

close(3)                                = 0

munmap(0x4a4000, 4096)                  = 0

open("/lib/libc.so.0", O_RDONLY)        = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=236388, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a4000

read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0j\0\1\0\0\0\220\273"..., 4096) = 4096

mmap2(NULL, 229024, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE|MAP_EXECUTABLE, 3, 0) = 0x580000

mmap2(NULL, 18184, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a8000

pread(3, ": accept: %m\n\0\0\0socket: protocol"..., 9664, 225280) = 9664

close(3)                                = 0

munmap(0x4a4000, 4096)                  = 0

open("/lib/libc.so.0", O_RDONLY)        = 3

fstat(3, {st_mode=S_IFREG|0755, st_size=236388, ...}) = 0

close(3)                                = 0

munmap(0x4a3000, 1183)                  = 0

stat("/lib/ld-uClibc.so.0", {st_mode=S_IFREG|0755, st_size=29824, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4a3000

ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0x4dfa00) = -1 ENOTTY (Inappropriate ioctl for device)

ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B115200 opost isig icanon echo ...}) = 0

getuid32()                              = 0

getgid32()                              = 0

setgid32(0)                             = 0

setuid32(0)                             = 0

mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|0x4000000, 0, 0) = 0x4e0000

getcwd("", 4098)                        = 2

lstat("proc", {st_mode=S_IFDIR|0755, st_size=1024, ...}) = 0

mount("proc", "/proc", "proc", MS_SILENT, "") = 0

_exit(0)

QuoteReplyEditDelete

 

 

2009-07-20 16:45:11     Re: Trouble mounting filesystems during startup

Mike Frysinger (UNITED STATES)

Message: 77769   

 

well about the next thing to try is to enable debug statements in the busybox mount source code to try and see where it's going wrong and why ...

Attachments

    Outcomes