Home - Writings - Pictures - Code - Resume - Links


Linux from Scratch: From "What's Bash?" to "dd if=/dev/random of=~/motherinlaw-0.0.95.tar.gz"

I recently posted to the group about having successfully created my own home-grown linux system, and there was response requesting a longer write-up of this brief period of lunacy I went through. Here goes, you asked for it.

First of all, a little background: I started with linux in, I think, 1998, when I found a copy of RedHat 4.1 in a bookstore in downtown Cairo. Yes, 4.1. And yes, it sucked. It took me 2 weeks to work out I had to recompile the kernel to get my sound card working. I spent countless hours cat'ing to /dev/ttyS0 hoping the kernel would get tired of my silly attempts and make it work by magic. X looked like something the Xerox research center at Palo Alto might have played with a bit in 1978. vi used strange keys, and it almost seemed like the thing was hacked to reallocate the keybindings on a pseudorandom basis. Like I said before, it sucked.

While asking around on IRC once about how to get a winmodem to work, it became apparent to me that 4.1 was not something I was supposed to have; some people on #linux suggested that I had inherited it, perhaps, from my grandfather. You know the people in #linux, quick, sharp tongues. The then-current version of RedHat was 6.x as I recall, and all of a sudden, in #linux, I felt like the salesman from the Mercedes car agency who drives home every night in a Fiat 128. A quick visit to http://www.redhat.com confirmed this, and I spent 2 years considering downloading ISO images on a 33600. I have successfully avoided bankruptcy, so you may assume I resisted the temptation to click on the download links.

I spent 2 or 3 fairly successful years in windows, up until the point where, upon browsing the linux HOWTO archive, I happened across the Linux from Scratch HOWTO. This wonderful document described a method to create your own linux system from nothing but source to create a proper working linux installation with very minimal overheads. "Heaven" I thought when I realized that this meant that I didn't need the whole RedHat ISO's, just the source packages needed for the system. These turned out to be somewhere between 70 and 80 Mb, very reasonable compared to 650 Mb. Needless to say, a few hours later the whole set of packages were sitting pretty on drive D.

Now, Linux from Scratch (LFS from now on) needs to be born out of an existing linux distribution, so I promptly dug out the old RedHat 4.1 discs from the garbage can and installed it. This was, I believe, back in November 2001. Without going into depressing and horrible details, the whole thing went nuts, with gcc spewing so many error messages that even /dev/null was complaining. A quick trip to #lfs on irc://irc.linuxfromscratch.org informed me that probably the RedHat 4.1 release was so horribly out of date that it had to be completely upgraded. This was my first ever encounter with the ugly face of Mr. Dependencies, and what a traumatic experience it was. I once again pointed my (windows based) browser at http://www.redhat.com and considered briefly going for the ISO's. Fortunately, however, I began to have visions of a phone bill with 42 zeroes on the right hand side and reconsidered.

One week later, while searching through google for variations line 'linux Egypt distributor sell buy', I cam across Mr. Bahram and his group, and the rest of that part of the story, as they say, is history.

After the meeting in the beginning of December 2001, I went home with fresh copies of RedHat 7.2 and its offspring, Mandrake 8.1, and life was good. You can imagine what the Mandrake installer will do to someone whose lasting impression of linux installation interfaces is recurring nightmares of his family rebuilt from scratch with ncurses linked in.

To cut a long story short, I had not one, but 2 very decent distros available to me; this begs a question: why LFS?

The answer to this question is not as simple as the question was, I think. While Mandrake and RedHat were quite lovely operating systems, there were things about them that simply rubbed me up the wrong way. When I first saw how far the linux desktop had come, I fully intended to switch to linux and KDE for my whole computing life. No more windows, I thought.

Seeing as I have a job in management at a very large local company, my first and overpowering requirement was an office suite. The best office suite out there for linux is at the moment, IMHO, openoffice.org. As most of you will testify, openoffice.org and StarOffice versions < 6.x are real pigs in operation. All the functionality is there, but it simply takes ages to load and run. Starcalc loaded in maybe 30 seconds under linux; under windows, Excel opened almost before I clicked on its icon. This soon began to bug me. And Kspread, Gnumeric, etc. all lacked functionality I consider vital; scripting languages, external data source support (preferably ODBC), you name it.

In addition, while linux is widely acknowledged as being one of the most stable solutions around, this only applies to pure console. KDE and, to a lesser extent, Gnome both suffer crashes. Not necessarily as many as windows 95, nor even anywhere as much, but crashes nonetheless. This meant that the factor of stability was no longer one I could count as a reason to switch permanently and exclusively to linux.

None of these arguments are new to you people here; we all know linux is not ready yet for the desktop. There was a very interesting discussion about this on http://slashdot.org. I'll try to find the link and post it here or in the group.

1 month after installing RedHat 7.2 on my box, I was therefore back in windows.

Then there was a submission on /. about some people working on a minimal resource linux distro; many of their reasons for doing so hit home with me, and it didn't take long for me to fish out the LFS-HOWTO again and give it another read. One thing you absolutely _must_ realize if you use linux is the following:

Just because the system works like this doesn't mean that it _has_ to.

Remember, this isn't Windows.

Let's think about this for a second; my RedHat installation boots in twice the time windows 98 boots. Does this have to be so?

Nope.

Gnome is a slow pig. Does this have to be so?

Nope.

I spent 45 minutes abusing netstat and nmap working out where the security problems were with my default RedHat install, then spent a further 30 minutes writing ipchains rules, flushing them, and rewriting them to lock down the machine. Did these servers have to be there? Did they have to be world-accessible to begin with? Did they have to come with promiscuous access rules as per default?

Nope, nope, and nope.

It should be becoming clearer by now why I went down the LFS road. Linux is not configurable, it's not even very configurable. It's _ridiculously_ configurable. You cannot even begin to imagine how configurable it is. And being the kind of person who doesn't like other people making assumptions about his preferences for him, there was only one thing a control-freak like yours truly could do.

Roll my own OS. - (cue theme from The Good, the Bad, and the Ugly)

Oh, and just for the record, there's one more significant reason I decided to make my own system which I have neglected to mention: I'm slightly mad. :)

The Ordeal

I'm not going to go into any sort of detail here; Mr. Beekmans has done a fine job in the HOWTO. I will, however, impart some impressions of the process as a real live person who survived a LFS experiment.

First of all, learn from my example; it's ok when all gcc generates is warnings, but don't ever let an error message go unchecked. I made this mistake with one small package, and at the end when I came to test my system, it booted alright but would not let me log in. For the clued in out there, it was the shadow package; I remembered vaguely that the installation procedure had gone wrong at some point, but I had ignored it and proceeded. Big. Mistake. Fortunately, linux is a modular system, and all I had to do was to recreate the shadow installation, duplicate the error, and track it down from there. Someone on the list thinks I'm a guru; it was in a post replying to my announcement that I had built my OS from source. Just to prove I'm still closer to lamer than guru, in the end, it turned out my mistake had been in a simple ln -sf, not even in the compile...

This is one thing you're gonna have to accept if you want to do LFS: you will make mistakes, and they will be stupid ones and they will be hard to track down sometimes. The trick is not to beat yourself senseless when you find out how stupid the mistake was. If you've ever read the story about the butterfly in Rio that flapped its wings, causing a snowstorm in Chicago, then you'll know how it feels when you miss out on one character in the install of one package, only to have this trip up another package 3 days later.

It's also worth keeping in mind that the HOWTO is not set in concrete; it is not perfect. There are always mistakes, bugs if you will, and as we come across them there are two things to do, one easy one less so. First of all, in the spirit of open source, you contact the author of the document/package and let them know what you found. Second and less easy is, you work your way around the problem. This can get a little hairy, like what I went through with texinfo.

Or should I say with texinfo and ./configure... This package, texinfo, simply _would_ _not_ compile without errors. LOTS of errors. I won't even begin to tell you how I busted my rear end trying to work out why texinfo didn't like my computer, but I will give a brief description of how I solved this problem to illustrate to you some possible scenarios you might face working on your LFS system.

I noticed that most of the errors referred to calls to undefined functions in a naughty file called terminal.c and through judicious use of apropos and man found out that these functions were standard to the ncurses library. Fair enough, I thought to myself, and installed ncurses on the host system - not the LFS system - after verifying that I didn't have it. A quick ./configure, make, and make install later and the errors were still there. Beginning to get a little annoyed, I went online and googled the problem. I came across some guy on a forum describing the same problem, of the linker not finding libncurses.so (IIRC). The solution, it seemed, was to mkdir /lib/ncurses && ln -sf to the .so file in /lib . This I did fully confident that I had the problem licked, and recompiled. The linker, however, didn't share my optimistic view of the problem and promptly gave me the exact same error messages as before. Seeing as the problem seemed confined to terminal.c, I decided that the way to go was to hack the source of the offender. Inside terminal.c I quickly found a couple of conditional includes to termcap.h . Aha, I thought, these must somehow be confusing the linker, I thought, and commented them out, leaving only a hardcoded #include . After making sure that termcap.h indeed existed in /usr/include I redid the install. Same errors. Anyhow, to make a long story short, in the end it turned out that the ./configure script simply needed to be forced to recache its settings; it was using a cached result from the found termcap.h test, which had been cached before I installed ncurses.

Still awake?

This is an example of the kind of persistence you will likely need to successfully complete an LFS install. The kind of problem-solving I described here is not the rule; by far most packages install peacefully, but then again, if they all installed without the need for manual trouble-shooting, it wouldn't be linux, would it?

It is this kind of problem-solving that will really teach you how a *nix box works; not changing the wallpaper under KDE. It is a kernel recompile gone bad that will teach you how much of your hardware works, not a system information dialog box under Gnome. Don't get me wrong, I'm all for the desktop environments, just not yet. I'm not going to wait for Linux to become ready for the desktop, I'm going to wait for KDE/Gnome to be ready for me. If you're not using the console (not Konsole either) IMHO, you're not using linux.

Anyhow, the point is here if you can do a simple './configure && make && make install' and you know a little about partitions, then you can probably do LFS. If you're patient and persistent you can probably do LFS.

I honestly can't think of a better method of learning how this little puppy ticks.

Next project: Assuming I can find myself some older hardware for cheap, a Beowulf cluster. Why? Because it's fun :)



Youssef M. Assad on 27-02-2002


This page is copyright Youssef M. Assad 2002. This article is freely redistributable in UNMODIFIED FORM, covering both the content and credits. If anyone's publishing it, I'd like to know about it first, however.