Gentoo to Ubuntu Migration Part 3: Conclusion

And now for the long-overdue and gripping conclusion of my Linux distribution journey. Over the last few posts, you've witnessed my curious initial dabblings with Slackware, my coming of age with Debian, and finally being all grown-up with Gentoo. I suppose that makes the Gentoo-to-Ubuntu switch my midlife crisis.

Cheesy metaphors aside, I should probably start out with a disclaimer that this isn't an anti-Gentoo post in anyway. My position on Linux distributions has always been that the best one is the one that works for you. In other words, it's a matter of personal preference. So this series is really a journey of how my preferences have evolved over time.

In a word, the biggest motivator for me was time. Gentoo is great but time consuming, both in terms of man-hours required to manage it, and the CPU time required for constant compiling. That may be a bit misleading to those who don't know Gentoo: once you have a stable setup, you really don't need to compile any additional packages (except security updates). However, I'm an obsessive tinkerer and upgrader, so I can't help but do an "emerge -vuD world" (i.e. update all packages on the system) every week or so.

The upside to having all packages built on your local machine is immense flexibility. This is supported via Gentoo's portage system, in particular, "USE" flags. USE flags typically correspond to build options passed to a source package's configure script. For anyone who's ever done a "./configure --help", you've probably noticed that most non-trivial packages have compile-time options. Typical examples include whether or not to build support for such things as: graphics, sound, internationalization/multi-lingual, different video and sound codecs, ssl, etc. An obvious example: if you have a server that has no connected monitor ("headless"), it's reasonable to build all your packages without X11 (Linux's core graphics API) or sound support.

As with many things that offer such flexibility, "with great power comes great responsibility". For myself, I took advantage of Gentoo's system, and had very specific configurations for each of my multiple machines. Sometimes I'd even do things like globally removing support for a feature, except for one or two packages. What this ultimately led to was tricky system updates: since so much open source software is "in-flight" (i.e. undergoing continuous development), package upgrades often introduced incompatibilities.

So, at least in my experience, at best half the time, I could get by with "emerge -vuD world", wait a few hours for all the new packages to recompile, and come back to a functioning, upgraded system. Unfortunately, the other half the time, I was tracking down "blocking" packages, playing version-specific games, tweaking config files, changing USE flags, etc. Consider, when there are dozens, sometimes hundreds of packages to be updated, all compiled from source, any one could break the build process---you can't really "set it and forget it". It requires some baby-sitting, or at least periodical checkups to make sure the upgrade is still progressing.

When I had more free time, I enjoyed this process. But my commute and job are leading to attention-starvation of all my hobbies, one of which is maintaining my Linux systems. (Which means, hopefully, someday I'll have the time to switch back to Gentoo.)

In short, I don't have the time for source-based distributions. Well, that's easy enough, as most distributions are binary-based anyway. But which one? I basically defaulted to Ubuntu: it's one of the more popular distributions, which should count for something. It's also based on Debian, with which I had previous experience, so, hopefully, I would find it semi-familiar. It also has role-specific flavors, i.e. Ubuntu Server (for servers), and MythBuntu (for MythTV)---in theory, I could have one operating system for all my computers, and quickly and consistently manage them.

So, given all that backstory and rationale, here I am today, running Ubuntu on my main workstation. I run Ubuntu Server on my file server/NAS box, as well as my backup server. But I still haven't completely switched: my firewall/gateway/NAT box runs OpenBSD, although I don't intend to ever change that. My parents' and my MythTV still run Gentoo. Since these systems are used by other people (my parents and wife, respectively), I didn't want to disrupt anything until I was sure I had a viable solution (plus my parents are 150 miles away!).

Overall, I feel I'm generally saving time using Ubuntu over Gentoo. But I won't go so far as to say it's been perfect. Even though I had experience with Debian (Ubuntu's baseline), there's still a learning curve. And as I've learned more about Ubuntu, I've developed a little collection of gripes (those will be the topic of some future blog post).

Gentoo to Ubuntu Migration Part 2: The College Years

As I'm sure last week's month's installment left you desperate for more...

I started college in May, 1997, and graduated in December, 2001---only one semester beyond the typical four years! I learned a lot in college. Obviously, there was the required "book learning" for my degree (computer science). I mostly enjoyed my classes, but I find book learning less engaging than hands-on learning.

The self-taught/hands-on learning, particularly with regards to Linux, is the topic of this post. Initially, I dual-booted Linux and Windows 95. I tried to use Linux whenever possible, but if I couldn't figure out how to do something, I reverted to Windows.

Sadly, my memory fails me as to my progression using Linux. But here are some highlights, in no particular order:

• My original distribution was Slackware. It's been far too long for me to make any kind of comment about its pros and cons. But I'm pretty sure that my installation was a mess before I ultimately switched to Debian. At the time, I didn't know anything about package management. Whenever I needed (or just wanted to play with) some piece of software, I would download it, and do the obligatory "./configure ; make ; make install".
• I remember learning about Samba, which allows Linux to access shared folders in Windows (and vice-versa). At the time, there weren't (or at least I couldn't find) any mature graphical "network neighborhood" browsers for Linux. So any file sharing I did was through the command-line smbclient.
• I learned about markup languages by writing papers in LaTeX. It took me a while to get past the non-WYSIWYG paradigm... but I eventually grew to love LaTeX, and markup languages in general, and still prefer them to WYSIWYG editors (however, the business world isn't so enlightened, and sadly I still have to interface with Microsoft Word in the "real world").
• Printing used to be a major hurdle in Linux. I have vague recollections of hacking the cryptic /etc/printcap file (I think that was it). I never actually learned the details of the file (e.g. its grammar and meaning), but mostly just looked for working config files on the Internet.
• Surfing the 'net: good old Netscape Navigator 4.x! This was about the time of the first "browser wars", i.e. crummy Netscape versus Microsoft's bundled-with-Windows Internet Explorer. I remember Netscape losing this battle, and Linux users were left with the increasingly obsolete and generally junky Navigator, waiting patiently (*really* patiently) for Mozilla to deliver a usable browser for Linux.
• I used FVWM (or maybe FVWM2, possibly FVWM95) as my window manager for a long time. At one point, I downloaded all of Gnome's libraries and dependencies, painstakingly compiled and installed each (in the right order of course), and... stuck with FVWM! To this day, I don't use the full-featured "desktop environments" (i.e. GNOME/KDE), but tend to stick with the lighter-weight stand-alone window manager. I recall playing with Enlightenment DR15, which had truly mind-blowing eye candy. But still, for whatever reason, stayed with FVWM.
• Editors. A classic Internet flame war. These days, and since senior year (maybe even junior year), I've been a die-hard vim user. But I did dabble with emacs for quite a while.
  

I wish I'd made a note of it (of course I never imagined myself blogging about it), but there came a point where I completely removed Windows from my PC. I think I did this around sophomore year, which would have been 1998 or 1999. I've literally been Microsoft-free ever since (at least on the computers I own). Basically, I reached a point where I was completely comfortable doing everything I needed or wanted to do under Linux. The Windows partition was increasingly becoming just wasted space, and I no longer felt I needed it as a safety net.

To this day, I wish I could remember exactly what prompted the complete changeover! I don't know if it was just a realization that I was spending over 99% of my time in Linux, or if there was some critical application for which I finally found an adequate substitution. Either way, the training wheels were formally off!

During these years, there were often "distribution wars". People vehemently supported their distribution; flame wars bashing each others' distributions were not uncommon. Being well-versed in Linux, but still not a true expert, I found myself consulting others as to which distribution I should choose. I went with Debian. Many others were supposedly easier to use, but most people said Debian made you do everything by hand, and was great if you wanted to learn about Linux system administration---which I did.

So Debian became my distribution of choice. This was before "apt-get"! I remember using the horrendous dselect program to manage packages. I haven't used that program in years, so perhaps it has improved, but at the time, I remember thinking it had about the worst user interface I'd ever encountered. (To be fair, perhaps it would make more sense to me now, given that I'm more familiar with the mindset of Linux developers, and common Unix idioms.)

I happily used Debian for many years. I let Debian's package manager (dpkg) manage all my programs. However, there were still many packages I needed (or just wanted to play with) for which a Debian package didn't exist, or existed but was too old. With some programs, I liked to be on the "bleeding edge". Debian's "stable" distribution is decidedly not bleeding edge---that is the price one pays for a stable and consistent system.

Vowing not to let my Debian installation degrade the way my Slackware distribution had, I vowed not to clutter my /usr/local area by blindly running through the "./configure ; make ; make install" routine. Enter what I still consider a really cool "secondary" package manager, epkg (the encap package manager). This program basically allowed you to self install your own programs into /usr/local/encap/packagename-version, and the program would manage symbolic links into the actual /usr/local/ directories.

As time went on, I grew tired of managing packages through epkg. The system worked, but it was somewhat tedious to self-manage an ever-increasing list of packages. Debian seemed "stuck", moving at a glacial pace. Some package versions were getting quite stale---epkg was needed not just for the bleeding edge, but even features that were added within the last year! Likewise, Debian packages were often compiled with features I didn't need, or compiled without features I wanted.

There had to be a better way... and that's when I found Gentoo. Again, I can't remember exactly when I discovered Gentoo. I believe it was after I had graduated college, probably around 2003 or 2004. Gentoo is unique in the Linux world, as it is one of the few source-based distributions. Instead of downloading and installing pre-built binaries, Gentoo's package manager downloads a program's source, builds it on the target machine, then installs the resulting program. The package management system, portage, is also a highly-configurable build system. Via the use of simple configuration files, it is quite easy to install and manage software packages that have been precisely tailored to your system.

Gentoo to Ubuntu Migration Part 1: Prologue

I've been using Linux for over a decade now, and loving almost every minute of it. Recently I changed the distribution I was using on my workstation from Gentoo to Ubuntu. Since one of the goals of this blog is to document my technology learning, I feel obliged to report on the transition experience. Also, at least one of my friends has an interest in this story.

In an attempt to make the story as dramatic as possible, I decided to break it into multiple sections. This, the first, is the back story: how I got into Linux.

I first got started with Linux around 1996, when I was still in high school. In BL Times (Before Linux), I ran a small bulletin board system (BBS) called raw sewage (a name which I cherished so much I use it for my website). The BBS is important because it gave me a requirement for my PC which was unusual at the time: multitasking, i.e. the ability to run multiple applications concurrently. These days, we take multitasking operating systems for granted, as virtually everybody has at least a couple programs running: a web browser, an email client, a spreadsheet or document, maybe a music player, and a game of solitaire when the boss isn't looking.

But my bulletin board days pre-dated Windows 95, which was Microsoft's first operating system that had semi-useful multitasking capabilities. (Yes, you could technically run multiple applications at once with Windows 3.x, but it was "fake" multitasking. I'm glossing over a lot of technical detail here: the techies should know what I mean and everyone else can take my word for it.)

Anyway, I was running MS-DOS, the old text-based operating system that required you to type in commands. DOS could only run one program at a time. I needed to have the bulletin board software running constantly, but I also wanted to be able to use my computer for other things. For a while, I used a program called DESQview, which effectively made DOS into a multitasking operating system. Eventually, for reasons I can't remember, DESQview fell out of favor with me.

I also dabbled in IBM's OS/2 Warp. I remember liking OS/2 for the most part. However, I ultimately gave it up as well, but, again, for reasons I can't remember. (My apologies: my memory is bad, and all this was over a decade ago---with college i.e. lots of beers---in between.)

At some point, I heard about this operating system called Linux that was a true multitasking system and free. Somehow, I happened across one of those multi-CD cases that contained at least three Linux distributions: RedHat, Debian and Slackware (there may have been more, but see my note about my memory). The case had a little insert that described how to install each operating system. I do remember that it suggested first-time users install Slackware: and so I did.

Of all the details I've forgotten, I do remember my initial reaction: underwhelmed. And I remember staying underwhelmed for quite a while, actually. Here I had read all these great things about Linux, how powerful it was, etc. And yet all I had was a foreign prompt staring at me. It resembled DOS, but it clearly wasn't DOS. It couldn't run my BBS software, or any other program with which I was familiar. And there was supposed to be a graphical interface, but I didn't know how to get to that... I remember I basically spent a lot of time just nosing around the file system, trying to make sense of things like "usr" and "var". But it was some time before I actually did anything useful with Linux.

MythTV Hardware Upgrade

When I started this blog, one of my goals was to consistently post any hardware or software problems I encountered while tinkering with my computers. This serves two purposes: if, in the future, I find myself having similar difficulties, I have a written record of what I did to refresh my memory (simply typing it all out will strengthen my memory). The other advantage is giving back to the community: so many times I've encountered a strange error or situation, only to find some obscure blog post that explains the exact same situation and solution. Perhaps some day a desperate computer enthusiast will use my blog to find and answer to their woes.

First, the short version: I need to pass the "pci=nomsi" parameter to the Linux kernel in order for my Asus M3N78-EM motherboard to recognize the hard drive.

Now the long version...

I'm a compulsive hardware tinkerer, always tweaking my computers, both software and hardware. I haven't bought a prebuilt computer for at least 15 years; all my computers were assembled piecewise by yours truly. Last week I was rebuilding my backup server (affectionately named "dumpster"), replacing the Intel CPU and motherboard with an AMD solution. My MythTV media PC ("cesspool") also used AMD. However, it had an older, slower CPU---a BE-2350---than the one I was about to put in the backup server, a 4850e. I thought to myself, why put the better processor in a machine that doesn't need it? Why not use the faster hardware for the machine that gets regular use?

There's one fact of which my wife loves to remind me; a fact I feel obligated to report in the interest of full-disclosure: there was absolutely nothing wrong with our MythTV prior to me attempting the CPU change; it didn't need a faster processor. But like I said, I have this compulsion: computers are toys to me, and I just want to play! Besides, what could go wrong with a simple, quick CPU swap?

I put the better CPU, the 4850e, in MythTV. I hit the power button and... nothing! The computer wouldn't POST (power on self-test). As I thought about what might be wrong, I wondered if the CPU was too new for the motherboard, a Biostar Tforce TF7025-M2. Some quick Google work confirmed my suspicions: Biostar's CPU support page doesn't list the 4850e as supported; likewise, Newegg's customer reviews say the same. D'oh!

Sad, but no problem: just put the old BE-2350 back, and finish the backup server... and it still won't POST! What happened? At this point, I assumed the motherboard was dead. (In hindsight, I don't know why I thought this.) I ordered a new board, an Asus M3N78-EM. This motherboard sports the nVidia GeForce 8300 video chip set.

When the new board arrived, I was excited: I get to use the 4850e (I checked teh M3N78's CPU compatibility before ordering), and would have all-around faster/newer hardware. So I dropped in the CPU, connected all the cables, hit the power button and... nothing. It wouldn't POST!

Now I was really broken. What was the problem? When a home built computer won't even POST, it could be one of any number of components: CPU, motherboard, memory, power supply. My first thought was that the case itself was causing a short. So I took the motherboard out of the case, set it on some cardboard (non-conductive), and tried again... No dice. Now I got to play the swap-one-component-at-a-time game to isolate the problem.

Foolish as I am, I tried everything in the wrong order: hardest-to-easiest. The simplest first check would be to swap RAM. But I went straight to disassembling the backup server to borrow its parts:

1. Tried a different power supply
2. Tried a different CPU (the old BE-2350, that I had to pull from the backup server)
3. Tried the 4850e in the backup server to verify the CPU wasn't dead

None of those worked, meaning, either the brand-new motherboard itself was dead, or the RAM was bad. I tried some different memory... success! After kicking myself for not trying the simpler troubleshooting first, I then kicked myself some more: I probably didn't even need a new motherboard in the first place. If only I had tried different RAM before impetuously ordering a new motherboard, I might not have had to deal with any of this mess!

On the other hand, I do have newer hardware that supports VDPAU, which is supported by MythTV.

Elated, I re-assembled the computer, hit the power button, and... now Linux fails to boot! Worse, my grub configuration specified the use of a splashimage, which wasn't found, resulting in garbled, unreadable boot text. I could make out enough of the text to tell that in mid boot there was a kernel panic, but couldn't discern the actual error. I then grabbed an Ubuntu installation/live CD. When it booted, I discovered that the hard drive wasn't being recognized. I turned to Google for my answer.

The second search result turned up this Ubuntu bug report. It talked about totally different hardware, by the symptoms were the same, basically:

ata1: SATA link up 3.0 Gbps (SStatus 13 SControl 30)
ata1.0: qc timeout (cmd 0xec)
ata1.0: failed to identify (I/O error, errmask=0x4)
ata1: failed to recover some devies, retrying in 5secs
ata1: SATA link up 3.0 Gbps (SStatus 13 SControl 30)
ata1.0: qc timeout (cmd 0xec)
ata1.0: failed to identify (I/O error, errmask=0x4)
ata1: failed to recover some devies, retrying in 5secs

The suggested workaround was simple enough: configure the SATA interface as AHCI in the BIOS (which I had already done), and pass the "pci=nomsi" option to the kernel at boot.

(Side note: that error message says "devies", shouldn't that be "devices"? Must have been a typo in that kernel version.)

I did exactly this, and... success! The system is now up and running. It hasn't even been 24 hours yet, but so far the system is stable.