So here's a quickie technical post. I powered on my Ubuntu 9.10-powered computer the other day. When I logged in, I found that my home directory didn't exist, and was dropped into the root directory.
I mounted it manually, and everything looked okay. Assuming it was just some fluke, I rebooted the machine to make sure it would auto-mount correctly. It failed again. This time, manual mounting didn't work.
My home directory actually lives on /dev/md0, which is a Linux software RAID1 (mirror) device. I checked, and the device existed, and the array state was good:
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[1] sdb1[0]
976759936 blocks [2/2] [UU]
unused devices:
Still, the manual mount failed:
# mount /dev/md0 /mnt/share
mount: /dev/md0 already mounted or /mnt/share busy
Huh? I ran the mount command with no options: neither /dev/md0 nor /mnt/share was listed anywhere. I ran lsof: neither of those files were listed as open.
I even tried mounting in a new directory:
# mkdir /mnt/tmp
# mount /dev/md0 /mnt/tmp
mount: /dev/md0 already mounted or /mnt/tmp busy
Well, clearly /mnt/tmp isn't busy, as it was just created! So what was wrong with md0?
At this point, I started to get a little worried, so did the sanity test to make sure my data was still there:
# mount -t ext3 /dev/sda1 /mnt/share
mount: /dev/sda1 already mounted or /mnt/share busy
I tried the same thing with /dev/sdb1, the mirror of sda1 in the RAID1. Same result. At this point, I was more than worried, but not yet panicked.
I checked dmesg, to see if there were any obvious problems. Nope. So then I went into /var/log, and started looking at those files for clues. I finally found something:
EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
Ah-hah! So I ran e2fsck on /dev/md0, and corrected the errors. I was then able to manually mount my home directory. I rebooted, again to make sure it came up, and it did.
I don't know if this is considered a mount bug or not, but the error message is extremely misleading.
This helped me out a lot. I had to remove the line for the array from fstab and reboot (even though it wasn't mounted!) before it would let me run fsck, though.
ReplyDeleteHad the same issue, but when I went to term 1 I was able to see that the system was running an fsck on /dev/md0 because max mounts had been hit. It's taking forever to get through it, heh.
ReplyDeleteHelps a lot !!
ReplyDeleteThanks
I got the same problem. however, i got device busy even when i run e2sfck
ReplyDelete/sbin/e2fsck /dev/sdb2
e2fsck 1.39 (29-May-2006)
/sbin/e2fsck: Device or resource busy while trying to open /dev/sdb2
and my system is running on sda
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 219G 1.4G 207G 1% /
/dev/sda2 4.8G 138M 4.4G 3% /tmp
tmpfs 3.7G 0 3.7G 0% /dev/shm
any idea??
I have 2 disks:
DeleteVolgroup00 - CentOS
VolGroup01 - Windows 7/ CentOS.
I booted with Volgroup 00 and I wanted to mount CentOS from VolGroup1 to fetch files there...
try following:
/sbin/pvscan
output should be something like:
PV /dev/sdb2 VG VolGroup01 lvm2 [xxx GB/0..
PV /dev/sda6 VG VolGroup 00 lvm2 [xxx...
finally
sudo mkdir /mnt/mymountpoint
mount /dev/VolGroup00/LogVol00 /mnt/mymountpoint
Thanks, this helped point me in the right direction.
ReplyDeleteHave been facing the same on our one client's server. Applied the steps and worked like a charm. Thanks Matt
ReplyDeleteThat's cool! Happy to hear about the update!
ReplyDeleteThanks for sharing this.
ReplyDelete