Device already mounted or mountpoint busy

First post in too long. The muse hasn't really grabbed me strongly enough to warrant additional entertainment for my tiny readership. Still, I won't give it up!

So here's a quickie technical post. I powered on my Ubuntu 9.10-powered computer the other day. When I logged in, I found that my home directory didn't exist, and was dropped into the root directory.

I mounted it manually, and everything looked okay. Assuming it was just some fluke, I rebooted the machine to make sure it would auto-mount correctly. It failed again. This time, manual mounting didn't work.

My home directory actually lives on /dev/md0, which is a Linux software RAID1 (mirror) device. I checked, and the device existed, and the array state was good:

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[1] sdb1[0]
      976759936 blocks [2/2] [UU]

      unused devices: 

Still, the manual mount failed:

# mount /dev/md0 /mnt/share
mount: /dev/md0 already mounted or /mnt/share busy

Huh? I ran the mount command with no options: neither /dev/md0 nor /mnt/share was listed anywhere. I ran lsof: neither of those files were listed as open.

I even tried mounting in a new directory:

# mkdir /mnt/tmp
# mount /dev/md0 /mnt/tmp
mount: /dev/md0 already mounted or /mnt/tmp busy

Well, clearly /mnt/tmp isn't busy, as it was just created! So what was wrong with md0?

At this point, I started to get a little worried, so did the sanity test to make sure my data was still there:

# mount -t ext3 /dev/sda1 /mnt/share
mount: /dev/sda1 already mounted or /mnt/share busy

I tried the same thing with /dev/sdb1, the mirror of sda1 in the RAID1. Same result. At this point, I was more than worried, but not yet panicked.

I checked dmesg, to see if there were any obvious problems. Nope. So then I went into /var/log, and started looking at those files for clues. I finally found something:

EXT3-fs warning: mounting fs with errors, running e2fsck is recommended

Ah-hah! So I ran e2fsck on /dev/md0, and corrected the errors. I was then able to manually mount my home directory. I rebooted, again to make sure it came up, and it did.

I don't know if this is considered a mount bug or not, but the error message is extremely misleading.

DHS = Paranoia

Yesterday, I was trying to take some artsy photos of the rail road tracks near my home, as well as some general autumn/changing leaves/nature shots. A local cop pulled up next to me and asked me about the pictures I was taking, in particular: "What are you taking pictures of?"

I answered, plainly, "Nature photos, the changing leaves, the railroad."

He then told me I looked suspicious. He went on to say that, with the Department of Homeland Security, taking pictures of railroads looks suspicious. He repeatedly used the word "suspicious".

He then asked if he could see the pictures I took. As I started thinking of a polite way to say "not without a warrant", he got paged on his CB radio. He chatted on the CB for a minute or so, then turned to me and gave another quick "it looks suspicious" speech, then drove away.

Overall, it wasn't a big deal. It only took up a few minutes of my time. The cop wasn't a jerk, simply matter-of-fact. I didn't have to give him my information (likewise, he didn't identify himself).

I have nothing against the police officer; he was just doing his job. What bothers me is that, a guy walking around with a camera in broad daylight is now considered "suspicious". Have we become so paranoid that hobbyist photographers are now included in the group of potential enemies?

The Most Annoying Office Mate Ever

I have a close friend who described to me, at length, the annoying habits of the person who sits closest to him at work. He went on in such detail that I felt like I was there...

The first and most notable annoying habit is the "violent nasal exhalation". Basically, we all have an opening that connects our nasal cavity to our mouth. Apparently, the annoying guy closes that off, builds up pressure in his mouth, then opens the nasal passage. The result is similar to the sound he would make if he was blowing his nose, but immediately preceded by a kind of "thunk" from the explosive opening of the mouth-to-nose cavity.

I've never heard a whale expel air through its blowhole, but I can only imagine it's very similar to what this guy does.

Of course, such a habit could easily be ignored if it occurred infrequently. My friend---who is never prone to exaggeration---says the guy makes the noise at least once every minute. Every minute!

The guy's next habit is less frequent, but louder and much more prominent: the "mouth vacuum". This involves opening the corner of his mouth and forcefully sucking in air. I've heard people (and probably done it myself) do this when their mouth is full of saliva. Of course, the silent alternative is just to close your mouth and swallow.

That he does it out of the corner of his mouth is important: the side he does this with is the same side on which my friend sits. It's as though the sound is directed squarely at him. It's quite loud too: loud enough that it can be heard in the whole office. But of course, my friend is at ground zero, where it's loudest and impossible to ignore. Frequency is lower than the nose thing, but still at least every five minutes.

Hacking and spitting. One to four times an hour, he'll hack up what ever drainage or phlegm he has in his sinuses, and spit it into his garbage can. As an added bonus, sometimes he'll skip the garbage can and use the sink in the office's kitchenette. My friend said that the other day, he heard the annoying guy hack and spit into the sink. That in and of itself was unremarkable, nothing new. But then my friend went over to the sink to rinse out a mug... and there it was, in all its phlegmy glory: a giant, yellowish ball of mucous, sitting innocently in the sink, half-way between the drain and the edge. He hadn't even bothered to rinse it down the drain! A public office sink turned spittoon.

He occasionally gets into throat-clearing fits, which becomes the culmination of all his foul habits. He'll sit there and continuously clear his throat for a solid minute or two, punctuating his clearings with violent nasal exhalations. Often a mouth vacuum or two is thrown in for good measure, and the spectacle's grand finale is hacking a big loogie into his garbage can. Fortunately, this only happens, on average, a couple times per day.

On days when my friend is really lucky (sarcasm), the guy will accompany every violent nasal exhalation with a throat-clearing. Like a lone barking dog, the sounds are too frequent to ignore, but not frequent enough to fade into the background.

My friend's office is small and generally quiet---whisper quiet, in fact, with an open seating plan (i.e. no cubes or anything dividing up one person's workspace from the next). This means that even the slightest of sounds can be heard.

The annoying guy uses this library-like silence to achieve his full annoyance potential: lip-smacking. Generally, such behavior is reserved for animals and sloppy-eating cartoon characters. But during, and for several minutes following any eating, the annoying guy manages to continuously slurp and smack his lips. If he only ate once or twice a day, perhaps these noises could be ignored. But most days, he eats at least six times: breakfast, lunch, two to four yogurt snacks, two to four nut snacks. And then there's the cookies: between all the other meals and snacks, he'll toss a whole cookie in his mouth, and just suck on it, presumably until it disintegrates. The sucking is of course open-mouthed, and practically at broadcast volume for my friend.

Also rising above the silence are random whisperings are finger snaps. Throughout the day, the annoying guy randomly whispers and snickers to himself. My friend doesn't know what causes the guy to talk to himself; presumably, he's thinking through something. But the giggling comes about when he's reading online forums and/or email. Once, the guy was so moved to laughter, that in an attempt to muffle the outburst, he actually produced the most awkward, hyena-like noise my friend has ever heard a human make. And if the random whisperings weren't distracting enough, they are often peppered by finger snaps. Yes, the guy snaps his fingers off and on throughout the day. It's not a constant snapping, just two or three snaps---as though he is trying to get someone's attention.

The solution to my friend's problems? McMaster-Carr part number 6207T53: taper end foam earplugs with a noise reduction rating (NRR) of 33 dB.

Gentoo to Ubuntu Migration Part 3: Conclusion

And now for the long-overdue and gripping conclusion of my Linux distribution journey. Over the last few posts, you've witnessed my curious initial dabblings with Slackware, my coming of age with Debian, and finally being all grown-up with Gentoo. I suppose that makes the Gentoo-to-Ubuntu switch my midlife crisis.

Cheesy metaphors aside, I should probably start out with a disclaimer that this isn't an anti-Gentoo post in anyway. My position on Linux distributions has always been that the best one is the one that works for you. In other words, it's a matter of personal preference. So this series is really a journey of how my preferences have evolved over time.

In a word, the biggest motivator for me was time. Gentoo is great but time consuming, both in terms of man-hours required to manage it, and the CPU time required for constant compiling. That may be a bit misleading to those who don't know Gentoo: once you have a stable setup, you really don't need to compile any additional packages (except security updates). However, I'm an obsessive tinkerer and upgrader, so I can't help but do an "emerge -vuD world" (i.e. update all packages on the system) every week or so.

The upside to having all packages built on your local machine is immense flexibility. This is supported via Gentoo's portage system, in particular, "USE" flags. USE flags typically correspond to build options passed to a source package's configure script. For anyone who's ever done a "./configure --help", you've probably noticed that most non-trivial packages have compile-time options. Typical examples include whether or not to build support for such things as: graphics, sound, internationalization/multi-lingual, different video and sound codecs, ssl, etc. An obvious example: if you have a server that has no connected monitor ("headless"), it's reasonable to build all your packages without X11 (Linux's core graphics API) or sound support.

As with many things that offer such flexibility, "with great power comes great responsibility". For myself, I took advantage of Gentoo's system, and had very specific configurations for each of my multiple machines. Sometimes I'd even do things like globally removing support for a feature, except for one or two packages. What this ultimately led to was tricky system updates: since so much open source software is "in-flight" (i.e. undergoing continuous development), package upgrades often introduced incompatibilities.

So, at least in my experience, at best half the time, I could get by with "emerge -vuD world", wait a few hours for all the new packages to recompile, and come back to a functioning, upgraded system. Unfortunately, the other half the time, I was tracking down "blocking" packages, playing version-specific games, tweaking config files, changing USE flags, etc. Consider, when there are dozens, sometimes hundreds of packages to be updated, all compiled from source, any one could break the build process---you can't really "set it and forget it". It requires some baby-sitting, or at least periodical checkups to make sure the upgrade is still progressing.

When I had more free time, I enjoyed this process. But my commute and job are leading to attention-starvation of all my hobbies, one of which is maintaining my Linux systems. (Which means, hopefully, someday I'll have the time to switch back to Gentoo.)

In short, I don't have the time for source-based distributions. Well, that's easy enough, as most distributions are binary-based anyway. But which one? I basically defaulted to Ubuntu: it's one of the more popular distributions, which should count for something. It's also based on Debian, with which I had previous experience, so, hopefully, I would find it semi-familiar. It also has role-specific flavors, i.e. Ubuntu Server (for servers), and MythBuntu (for MythTV)---in theory, I could have one operating system for all my computers, and quickly and consistently manage them.

So, given all that backstory and rationale, here I am today, running Ubuntu on my main workstation. I run Ubuntu Server on my file server/NAS box, as well as my backup server. But I still haven't completely switched: my firewall/gateway/NAT box runs OpenBSD, although I don't intend to ever change that. My parents' and my MythTV still run Gentoo. Since these systems are used by other people (my parents and wife, respectively), I didn't want to disrupt anything until I was sure I had a viable solution (plus my parents are 150 miles away!).

Overall, I feel I'm generally saving time using Ubuntu over Gentoo. But I won't go so far as to say it's been perfect. Even though I had experience with Debian (Ubuntu's baseline), there's still a learning curve. And as I've learned more about Ubuntu, I've developed a little collection of gripes (those will be the topic of some future blog post).

The Irony of my Blog Name

It recently occurred to me that, recently, my blog's title is kind of ironic. I originally wrote it from my perspective, as a bit of cliche, self-deprecating humor: I'm the stand-up comedian getting no response from the crowd. And that's because I had low expectations for the size of this blog's audience.

So I went well over a month without any posts, the one or two readers I might have were asking the same thing: is this thing on?

Passion in Jobs

Roughly a year ago, I started an essay called Passion vs. Engagement. The piece is unfinished, but so are my thoughts on the matter; I'm still thinking about this stuff, and how I want to present it... at least, how to present it in a formal, professional tone. But this blog is neither formal nor professional; it is a playground for thinking out-loud.

Anyway.

As my line of work is software development (aka "programming" aka "coding"), I find myself reading Joel Spolsky's blog from time to time. Though I have no desire to move to New York, the work environment for his programmers is envious: individual offices, comfortable/ergonomic work spaces, modern tools and equipment, etc. He's also hinted to the fact that pay at his company is very competitive (although, personally, I'd take a cut in pay to actually have a quiet, distraction-free environment).

However, he eludes to passion for programming as effectively being a hiring criterion. I don't think Joel is any different from virtually every recruiter, HR staffer, or hiring manager (in any field) when he says he wants passionate employees. Who doesn't?

But my question is, what exactly is passion? Joel gives some hints to what he means by passion: evidence of programming-related pursuits outside of work. Although I genuinely love programming, I have to admit: after 11 hours of it, each and every day, I have no desire to write any code outside of work. I have too many other hobbies that are already on the brink of attention starvation. Does that mean I'm not passionate?

I can think of a few examples where I've witnessed true passion:

1. My friend and boss where I'm currently employed. Relentless 11--12 hour days, with practically no break in focus. While I'm in the same boat, after three years, I already know this isn't something I can sustain for the long haul. But my boss has been working like this for well over a decade; my friend is approaching a decade of the same. It's neither an exaggeration nor an unfair statement to say that both would work even more hours if it weren't for the wife and kids.
2. My wife used to work for a non-profit organization. Due to matters very close to her heart, this was a cause for which she was definitely passionate. When she worked for this company, she worked as many or more hours than I do; and they were erratic hours at that (nights, weekends, etc). Being a non-profit, her pay was substantially below her effort.
3. There are at least a couple open-source developers whose lives I've researched a bit. These people develop software for their employer by day, and write just as much open-source code by night.

In all cases, the common trait is a singular pursuit in a given field.

But for me, I simply have too many interests to devote myself entirely to one.

I think (or at least I'd like to think) that most people are more like me---we willingly fall into the "work-to-live" camp (as opposed to the "live-to-work" one). That is, I'd wager that the majority of people, even the ones who love their jobs, want to have time to do other things besides work.

In other words, if my guess is correct, most people aren't passionate about their work.

And maybe that's just it: the majority of employers aren't looking for "most people"; they want the best (who are, by definition, a minority). But the main idea behind my essay (and this blog post) is that I believe there exists a happy medium between passionate and mediocre. I call it being engaged; it's Passionate, Jr., but still mutually beneficial to employer and employee. Engagement has most of the attributes of passion, but stops short of being a singular life focus.

Gentoo to Ubuntu Migration Part 2: The College Years

As I'm sure last week's month's installment left you desperate for more...

I started college in May, 1997, and graduated in December, 2001---only one semester beyond the typical four years! I learned a lot in college. Obviously, there was the required "book learning" for my degree (computer science). I mostly enjoyed my classes, but I find book learning less engaging than hands-on learning.

The self-taught/hands-on learning, particularly with regards to Linux, is the topic of this post. Initially, I dual-booted Linux and Windows 95. I tried to use Linux whenever possible, but if I couldn't figure out how to do something, I reverted to Windows.

Sadly, my memory fails me as to my progression using Linux. But here are some highlights, in no particular order:

• My original distribution was Slackware. It's been far too long for me to make any kind of comment about its pros and cons. But I'm pretty sure that my installation was a mess before I ultimately switched to Debian. At the time, I didn't know anything about package management. Whenever I needed (or just wanted to play with) some piece of software, I would download it, and do the obligatory "./configure ; make ; make install".
• I remember learning about Samba, which allows Linux to access shared folders in Windows (and vice-versa). At the time, there weren't (or at least I couldn't find) any mature graphical "network neighborhood" browsers for Linux. So any file sharing I did was through the command-line smbclient.
• I learned about markup languages by writing papers in LaTeX. It took me a while to get past the non-WYSIWYG paradigm... but I eventually grew to love LaTeX, and markup languages in general, and still prefer them to WYSIWYG editors (however, the business world isn't so enlightened, and sadly I still have to interface with Microsoft Word in the "real world").
• Printing used to be a major hurdle in Linux. I have vague recollections of hacking the cryptic /etc/printcap file (I think that was it). I never actually learned the details of the file (e.g. its grammar and meaning), but mostly just looked for working config files on the Internet.
• Surfing the 'net: good old Netscape Navigator 4.x! This was about the time of the first "browser wars", i.e. crummy Netscape versus Microsoft's bundled-with-Windows Internet Explorer. I remember Netscape losing this battle, and Linux users were left with the increasingly obsolete and generally junky Navigator, waiting patiently (*really* patiently) for Mozilla to deliver a usable browser for Linux.
• I used FVWM (or maybe FVWM2, possibly FVWM95) as my window manager for a long time. At one point, I downloaded all of Gnome's libraries and dependencies, painstakingly compiled and installed each (in the right order of course), and... stuck with FVWM! To this day, I don't use the full-featured "desktop environments" (i.e. GNOME/KDE), but tend to stick with the lighter-weight stand-alone window manager. I recall playing with Enlightenment DR15, which had truly mind-blowing eye candy. But still, for whatever reason, stayed with FVWM.
• Editors. A classic Internet flame war. These days, and since senior year (maybe even junior year), I've been a die-hard vim user. But I did dabble with emacs for quite a while.
  

I wish I'd made a note of it (of course I never imagined myself blogging about it), but there came a point where I completely removed Windows from my PC. I think I did this around sophomore year, which would have been 1998 or 1999. I've literally been Microsoft-free ever since (at least on the computers I own). Basically, I reached a point where I was completely comfortable doing everything I needed or wanted to do under Linux. The Windows partition was increasingly becoming just wasted space, and I no longer felt I needed it as a safety net.

To this day, I wish I could remember exactly what prompted the complete changeover! I don't know if it was just a realization that I was spending over 99% of my time in Linux, or if there was some critical application for which I finally found an adequate substitution. Either way, the training wheels were formally off!

During these years, there were often "distribution wars". People vehemently supported their distribution; flame wars bashing each others' distributions were not uncommon. Being well-versed in Linux, but still not a true expert, I found myself consulting others as to which distribution I should choose. I went with Debian. Many others were supposedly easier to use, but most people said Debian made you do everything by hand, and was great if you wanted to learn about Linux system administration---which I did.

So Debian became my distribution of choice. This was before "apt-get"! I remember using the horrendous dselect program to manage packages. I haven't used that program in years, so perhaps it has improved, but at the time, I remember thinking it had about the worst user interface I'd ever encountered. (To be fair, perhaps it would make more sense to me now, given that I'm more familiar with the mindset of Linux developers, and common Unix idioms.)

I happily used Debian for many years. I let Debian's package manager (dpkg) manage all my programs. However, there were still many packages I needed (or just wanted to play with) for which a Debian package didn't exist, or existed but was too old. With some programs, I liked to be on the "bleeding edge". Debian's "stable" distribution is decidedly not bleeding edge---that is the price one pays for a stable and consistent system.

Vowing not to let my Debian installation degrade the way my Slackware distribution had, I vowed not to clutter my /usr/local area by blindly running through the "./configure ; make ; make install" routine. Enter what I still consider a really cool "secondary" package manager, epkg (the encap package manager). This program basically allowed you to self install your own programs into /usr/local/encap/packagename-version, and the program would manage symbolic links into the actual /usr/local/ directories.

As time went on, I grew tired of managing packages through epkg. The system worked, but it was somewhat tedious to self-manage an ever-increasing list of packages. Debian seemed "stuck", moving at a glacial pace. Some package versions were getting quite stale---epkg was needed not just for the bleeding edge, but even features that were added within the last year! Likewise, Debian packages were often compiled with features I didn't need, or compiled without features I wanted.

There had to be a better way... and that's when I found Gentoo. Again, I can't remember exactly when I discovered Gentoo. I believe it was after I had graduated college, probably around 2003 or 2004. Gentoo is unique in the Linux world, as it is one of the few source-based distributions. Instead of downloading and installing pre-built binaries, Gentoo's package manager downloads a program's source, builds it on the target machine, then installs the resulting program. The package management system, portage, is also a highly-configurable build system. Via the use of simple configuration files, it is quite easy to install and manage software packages that have been precisely tailored to your system.