A Minimal Linux System From Scratch

News

A Minimal Linux System From Scratch


« Back to news

Well after toying with the idea for a while I thought I'd start writing an article or maybe even articles (shock, horror!) that you will hopefully find informative and interesting. As I've been messing around with Linux recently I thought I'd start with giving you some basic information on how to set up you own minimal Linux system/embedded system from scratch using Qemu.

To get your tiny and ultra customised system off the ground you'll initially need the following:

This article is divided into the following sections:

  1. Creating a virtual disk image
  2. Setting up the partitions filesystem
  3. Getting a boot loader installed
  4. Booting the kernel
  5. Creating a Linux file system
  6. Further reading
  7. Troubleshooting
  8. Contact

1. Creating a virtual disk image

2. Setting up the partitions filesystem

To create the file system you need to know how many blocks the partition uses i.e. how much space the partition uses in the disk image. This is especially important if the partition doesn't completely the fill disk image (one reason could be because if it did, it wouldn't end on a cylinder boundary and thus be unmountable/unusable). This will be explained further once other topics have been covered.


Why is the partition block size needed?

As mentioned above, it's very important to specify the block size when creating the file system. This may seem strange as you wouldn't normally need to if you were formatting a normal hard disk. The reason (hopefully) becomes somewhat clear when you look at the following:

File system corruption without specifying block size

The mounted images partition table is ignored so as far as any tool like mke2fs is concerned, the partition is as big as the image size minus the partition table size. If the partition had to be smaller than the total available space so that it would end on a cylinder boundary or if you specified a smaller value for its size then mke2fs is wrong and there will be unused space which must not be used. The red arrow highlights what could happen if it is used, a file could be scattered across the image so that one of its blocks is located in the unused/invalid space. When this disk image is booted in Qemu and the block accessed, Linux will detect that it's outside the partitions limit and will throw up nasty errors such as "attempt to access beyond end of device" as shown here:

kernel error showing access beyond end of device
(This can also happen when using GRUB and manifests itself as the dreaded "error 24: Attempt to access block outside partition" message)

The worst part in all this is that you may be lucky and your initial system may have all its files located completely within the valid partition only to fail later when you start to add more data.

Under the hood of mke2fs

In case you want to take a look at the code yourself, you can download version 1.40.2 of e2fsprogs here which includes mke2fs.

In this example run through I am presuming mke2fs has been executed without a block count specified. I'll start by looking at misc/mke2fs.c at line 1238 which calls ext2fs_get_device_size. This gets the devices size in blocks, so for an 8MB image (8388608 bytes) it will first discover that its size in bytes is 8356352 which is the total size minus the partition table size. It then gets the number of blocks by dividing this value by the block size (which was specified with the -b option as 1024) giving 8160.5. Because of integer rounding this becomes 8160. For those that prefer code, a significantly cleaned up version of the function looks like the following:

    1 errcode_t ext2fs_get_device_size(

    2                             const char *file, /* = /dev/loop0 */

    3                             int blocksize,    /* = 1024 because of –b */

    4                             blk_t *retblocks) /* = 0 at the moment */

    5 {

    6 

    7     int            fd, rc = 0;

    8     unsigned long long size64;

    9 

   10     /* opens /dev/loop0 and stores the descriptor in fd */

   11     fd = open( file, O_RDONLY );

   12 

   13     /* Gets the size of the disk image minus partition table size*/

   14     ioctl( fd, BLKGETSIZE64, &size64 );

   15     /* size64 is now 8356352 */

   16 

   17     *retblocks = size64 / blocksize;

   18     /* *retblocks = 8160 */   

   19 

   20     close( fd );

   21     return rc;

   22 }

(This beautiful code was made possible by CodeSourceAsHTML)

Once this returns, nothing else relevant happens until line 1276 which looks like the following:

    1 if (sys_page_size > EXT2_BLOCK_SIZE(&fs_param))

    2         /* Currently, fs_param.s_blocks_count = 8160 */

    3         fs_param.s_blocks_count &= ~((sys_page_size /

    4                                     EXT2_BLOCK_SIZE(&fs_param))-1);

The first test will compare 4096 (this is the default page size used in the kernel on most home computers) represented by sys_page_size with 1024. 1024 comes from following macros from lib/ext2_fs.h:

    1 #define EXT2_MIN_BLOCK_LOG_SIZE    10

    2 

    3 #define EXT2_MIN_BLOCK_SIZE        (1 << EXT2_MIN_BLOCK_LOG_SIZE) /*This gives you 1024*/

    4 

    5 #define EXT2_BLOCK_SIZE(s)        (EXT2_MIN_BLOCK_SIZE << (s)->s_log_block_size)

    6 /*Because its ext2, s_log_block_size is 0 thus EXT2_MIN_BLOCK_SIZE is unaffected and remains at 1024*/

Obviously 4096 is greater than 1024 so on the next line you can see fs_param.s_blocks_count (currently set to 8160) which gets manipulated so that the blocks on the file system will fit efficiently into memory. Translating the code to numbers gives you:

	fs_param.s_blocks_count &= ~((sys_page_size /
                            EXT2_BLOCK_SIZE(&fs_param))-1);

            → 8160 &= ~((4096 / 1024)-1);
            → 8160 &= ~3;
            → 1111111100000 (8160) &
              1111111111100 (~3)
               =  1111111100000 (8160)

In this case nothing happens because 8160 is already a multiple of 4.

Breaking "~((4096 / 1024)-1);" down, you can ignore the -1 and logical not (~) as all that does is help to convert the number (in this case 8160) to a multiple of whatever it needs to be. The important part is the 4096 / 1024 which determines what the multiple is. Now, if you don't know what a page size is you can basically think of it as a value that divides up memory into chunks. In this case the chunks would be 4096 bytes. This calculation ensures that if the entire disk was loaded into RAM, every chunk used would be completely filled making the most efficient use of memory. You can see this by doing the following calculation 8160 * 512 = 4177920 bytes. Loaded into RAM this would take up exactly 4177920/4096 = 1020 chunks

3. Getting a boot loader installed

Although there are other boot loaders, these days GRUB is almost the standard for UNIX based systems so if you haven't already got it then download version 0.97 from here and configure, make and install it. If you have an old distribution it might be useful to get an updated version and install it to a different directory using the -prefix=/DIRECTORY switch on configure.

4. Booting the kernel

GRUB is now installed, all it needs is something to boot i.e. a kernel. You can download and compile your own, use the one that came with your distribution, its up to you but however you find one, you'll want to copy it into the hard disk images boot directory (don't forget to remount your image with the -o parameter).

IMPORTANT: Unless you're using an initrd image, you must make sure that your kernel has compiled in support for the file system on your root partition (in this case ext2). You must also make sure you have the appropriate IDE drivers compiled in which includes the following options in the kernel configuration menus:

→ Device Drivers                                                                                         
             → ATA/ATAPI/MFM/RLL support                                             
                  → Enhanced IDE/MFM/RLL disk/cdrom/tape/floppy support
               → Include IDE/ATA-2 DISK support
                       → Generic/default IDE chipset support

In this case I've copied a kernel image called vmlinux-2.6.21-1 which you can get from here.

GRUB now needs to be told to boot this image so fire up your favourite text editor and open the grub.conf config file which you created earlier (found in boot/grub) and enter the following:

default 0
        timeout 30

        title Linux-2.6.21-1
        root (hd0,0)
        kernel /boot/vmlinux-2.6.21-1 root=/dev/hda1 rw

If you want more information on the configuration file syntax, have a look here

Now that the kernel is setup and booting its time to remount the image and add the minimal amount of user space applications that will present a command line.

5. Creating a Linux file system

No minimal Linux system would be complete without busybox. It provides all the standard tools you might need e.g. listing directories, copying files etc all in one file reducing the executable file header overhead.

Now you have the smallest bootable Linux system that could possibly exist (aside from cutting stuff out of the kernel!) up and running. It's not secure and can't do much but its certainly lean and will only get as fat as you want it to, its also useful if you want to do some kernel debugging as you wont have unnecessary processes etc polluting your data structures and generally getting in the way.

So go forth and create your own distribution and while your at it let me know if you found this useful. Enjoy!

6. Further reading

http://www.linuxfromscratch.org - setting up a complete Linux system from scratch
http://free-electrons.com/articles/elfs - more technical information on setting up a minimal system
Embedded Linux Primer: A Practical, Real-World Approach - provides a good overview of many aspects for a minimal Linux system
http://axiom.anu.edu.au/~okeefe/p2b/buildMin/buildMin.html - another old but useful minimal Linux setup

7. Troubleshooting

Q: On boot up I get the message "No init found"
A: Make sure your kernel has compiled in support for IDE and your root file system (unless your using initrd), check that the permissions are set correctly on all executables and libraries or just run chmod -c 755 FILENAME on everything other than the console device file, check for any other errors further up in the kernel log and also make sure you set up your file system properly (see below).

Q: I get error 24 when trying to use grub and/or the kernel gives me the error "attempt to access beyond end of device"
A: Something must have gone wrong when you set up your hard disk image, unfortunately this means you'll have to go through all the steps again. You probably want to pay particular attention to the steps entitled Creating a virtual disk image and Setting up the partitions filesystem.

8. Contact

If you have any questions, comments etc please feel free to email me. All my details can be found here.