Monday, December 7, 2009

Device already mounted or mountpoint busy

First post in too long. The muse hasn't really grabbed me strongly enough to warrant additional entertainment for my tiny readership. Still, I won't give it up!

So here's a quickie technical post. I powered on my Ubuntu 9.10-powered computer the other day. When I logged in, I found that my home directory didn't exist, and was dropped into the root directory.

I mounted it manually, and everything looked okay. Assuming it was just some fluke, I rebooted the machine to make sure it would auto-mount correctly. It failed again. This time, manual mounting didn't work.

My home directory actually lives on /dev/md0, which is a Linux software RAID1 (mirror) device. I checked, and the device existed, and the array state was good:

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[1] sdb1[0]
976759936 blocks [2/2] [UU]

unused devices:

Still, the manual mount failed:

# mount /dev/md0 /mnt/share
mount: /dev/md0 already mounted or /mnt/share busy

Huh? I ran the mount command with no options: neither /dev/md0 nor /mnt/share was listed anywhere. I ran lsof: neither of those files were listed as open.

I even tried mounting in a new directory:

# mkdir /mnt/tmp
# mount /dev/md0 /mnt/tmp
mount: /dev/md0 already mounted or /mnt/tmp busy

Well, clearly /mnt/tmp isn't busy, as it was just created! So what was wrong with md0?

At this point, I started to get a little worried, so did the sanity test to make sure my data was still there:

# mount -t ext3 /dev/sda1 /mnt/share
mount: /dev/sda1 already mounted or /mnt/share busy

I tried the same thing with /dev/sdb1, the mirror of sda1 in the RAID1. Same result. At this point, I was more than worried, but not yet panicked.

I checked dmesg, to see if there were any obvious problems. Nope. So then I went into /var/log, and started looking at those files for clues. I finally found something:

EXT3-fs warning: mounting fs with errors, running e2fsck is recommended

Ah-hah! So I ran e2fsck on /dev/md0, and corrected the errors. I was then able to manually mount my home directory. I rebooted, again to make sure it came up, and it did.

I don't know if this is considered a mount bug or not, but the error message is extremely misleading.

9 comments:

  1. This helped me out a lot. I had to remove the line for the array from fstab and reboot (even though it wasn't mounted!) before it would let me run fsck, though.

    ReplyDelete
  2. Had the same issue, but when I went to term 1 I was able to see that the system was running an fsck on /dev/md0 because max mounts had been hit. It's taking forever to get through it, heh.

    ReplyDelete
  3. Helps a lot !!

    Thanks

    ReplyDelete
  4. I got the same problem. however, i got device busy even when i run e2sfck

    /sbin/e2fsck /dev/sdb2
    e2fsck 1.39 (29-May-2006)
    /sbin/e2fsck: Device or resource busy while trying to open /dev/sdb2


    and my system is running on sda
    df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda1 219G 1.4G 207G 1% /
    /dev/sda2 4.8G 138M 4.4G 3% /tmp
    tmpfs 3.7G 0 3.7G 0% /dev/shm

    any idea??

    ReplyDelete
    Replies
    1. I have 2 disks:
      Volgroup00 - CentOS
      VolGroup01 - Windows 7/ CentOS.

      I booted with Volgroup 00 and I wanted to mount CentOS from VolGroup1 to fetch files there...

      try following:

      /sbin/pvscan

      output should be something like:

      PV /dev/sdb2 VG VolGroup01 lvm2 [xxx GB/0..
      PV /dev/sda6 VG VolGroup 00 lvm2 [xxx...

      finally

      sudo mkdir /mnt/mymountpoint

      mount /dev/VolGroup00/LogVol00 /mnt/mymountpoint

      Delete
  5. Thanks, this helped point me in the right direction.

    ReplyDelete
  6. Have been facing the same on our one client's server. Applied the steps and worked like a charm. Thanks Matt

    ReplyDelete
  7. That's cool! Happy to hear about the update!

    ReplyDelete