Retinix
News
About Retinix
Articles
Projects
Links

Minimal Linux Proxy

Introduction

I've always liked the idea of having a seperate internet gateway system or proxy server. When you have a few systems networked together, such a setup is quite useful. I recently upgraded to a higher speed ISP connection. It is a terrestrial microwave connection. The basics of it are:
  • You have a PC with a modem and an internal satellite card.
  • The modem dials the ISP and connects as usual. All outbound requests go via the modem. However, most inbound traffic is sent back to you via a dish on the side of your house ... and then into your satellite card (I think DNS lookups actually come back via the modem though). This gives you high download speed and crappy modem speed for uploads.
The use of an internal satellite card was bound to make this difficult to do in Linux but the card manufacturer Telemann do make Linux drivers available. In addition to that I wanted to use an internal Lucent Winmodem for my modem. If this complicated setup weren't enough, I also wanted to achieve this proxy on an old P75 with 16MB and a 500mb hard disk. Impossible?? (I think Lucent recommend at least a P133).

First steps

Initially I loaded Slackware 7.1 onto the P75 system. I prefer Slackware and you can be quite selective about what gets installed. I loaded the basics of a console only system (no X) with gcc. I think it came in around 200mb from memory. From their I knew I was going to have to update the kernel. I downloaded the 2.2.16 kernel since I knew that I could set up the satellite card drivers for it. After the kernel compile I was up close to 300mb. The P75 initially had an ISA 3c509 LAN card, but has since had a PCI 100Mbps RTL8139 put in (as well).

I used the freely available 5.68 Lucent modem driver. Now this does not work on kernels above 2.2.13 (or .14??), so I had to modify the tty.h include in the kernel, and move the poll_wait to the end of the structure.

I set up the LAN cards and ppp driver to load as modules. The sm200d (satellite card) driver loads as a module and the Lucent driver as well.

I won't go on any further about the ISP specific setup, suffice to say I got it all going. I even set up pppd demand dialling so I just had to ping a remote address and the modem would start dialling. I also set up ipchains so that it would restrict access across the gateway (now firewall).

Towards a minimal install

The Slackware gateway I'd setup was fine. It did the job and seemed to work OK. However, there were a few further goals to achieve:
  • Use something other than ext2 for the filesystem so that you can just 'turn it on and off'.
  • Cut down the size of the required disk space, just so that its easy to rebuild.

An alternative file system

I had pretty much made my mind up to try ReiserFS with this. With its journalling, I thought you should theoretically be able to just turn it off and the next time it starts, it rolls back and does what it needs to do (quickly) to get the FS to a consistent state.

Being a 2.2.16 kernel, I needed to directly patch the kernel using the patches from the ReiserFS site. I also downloaded and compiled the ReiserFS utils so that I could do a 'mkreiserfs'. My 500mb hard drive was filling up quickly with all this compiling, but I was able to use GNU Parted to shrink the main ext2 partition down by 50mb so I could create my new ReiserFS partition to play with.

The first thing to do was to format my new 50mb partition with mkreiserfs. I did this, but got quite a surprise when I mounted it for the first time. Out of my 50mb partition only 20mb was available for files. 30mb had been chewed up by ReiserFS's 'overheads'. I accepted this as the cost of having journalling and moved on. I also guessed that the total filespace requirement for my gateway should be quite a bit less than 50mb anyway.

Creating the reduced build

Having mucked around with building my own distribution a bit, I knew quite well what was required to get a Linux system 'bootable'. At this point the easiest way was to use my existing setup in Slackware as a base and just copy across the required files.

To simplify I copied my entire /lib tree and /dev tree and then hand picked files from /bin, /usr/bin, /sbin and /etc as required. An important consideration was to setup the correct directory structures under /var (eg. /var/lock to allow pppd to do locking correctly).

At the end of my hand copying I had only used up about 10mb. If I took more time and used small versions of certain files (eg. Busybox versions), then I'm sure I could get it much smaller. Since I'm only experimenting at this stage it would have to do.

I had some fun initially getting it to boot. I had the requisite LILO 21-6.1 (??). I can't quite remember what I did, but I got it to boot and it all seemed to go quite well.

ReiserFS log rollbacks!

Unfortunately, my ideas of simply turning it off and on at my leisure weren't quite successful. Sure I could turn it off and it would start up correctly after I turned it on, BUT ReiserFS decided to take quite a long time to rollback and correct itself. I had always thought that Journalling filesystems could rollback quickly and get going. Not in my case. With my setup, a crappy old 500mb harddrive and slow P75, the rolling back seemed to be taking forever. It would have been just as quick having a ext2 filesystem fsck itself.

Using a read-only Filesystem

Given that my ReiserFS idea wasn't so succesful, I decided on trying an ext2 filesystem, but only mounting it read-only. This way there would definately be no file damage when I turned it off. Even though logging to disk would be nice, in the multitude of uses, I thought no logging would be fine.

Problems with running Linux read-only

A complete Linux system doesn't really like having no write access to its filesystems. The main case for this is device files. For example, when a user logs in on the tty1 virtual console on a Linux system, the login process tries to 'chown' the /dev/tty1 device to the user that is logging in. If the /dev/tty1 file is in a read-only filesystem, the user won't be allowed to log in. Quite a problem really. Another case is the syslogd daemon. When it starts up it actually creates the /dev/log device (which is obviously not possible in a read only file system. And I knew that pppd usually likes to create a lock file under /var/lock.

Using a partial ram disk

My solution to all this was the following:
  • Use the same basic file structure as my ReiserFS based build
  • Use an ext2 filesystem for /
  • Leave the / filesystem mounted read-only when the system comes up (ie. don't remount / read/write)
  • Create a small RAM disk as the system boots and mount it as /var
  • Create the appropriate directory structure under /var
  • Create a /var/dev to put specific read/writable device files into
  • Replace the read/write devices with soft-links into /var/dev (this needs to be done from a seperate session prior to the first boot). 
         eg.
     ls -l /dev/tty1
      lrwxrwxrwx   1 root root  13 Apr  6 10:08 /dev/tty1 -> /var/dev/tty1

     ls -l /var/dev/tty1
      crw--w--w-   1 root     root       4,   1 Apr  7 11:13 /var/dev/tty1
  • Similarly for /dev/log, Just create a soft-link in /dev, but use the -p option of syslogd in order to create its device file in a different place. 
          eg.
      ls -l /dev/log
       lrwxrwxrwx   1 root root   12 Apr  6 10:55 /dev/log -> /var/dev/log

      Start syslogd as:
       /sbin/syslogd -p /var/dev/log

Success

Using the read-only filesystem setup works quite well. I still fsck the filesystem as the system boots, but it never finds any faults so it boots quite quickly.

Directory trees for the Read-Only Build

/dev

lrwxrwxrwx   1 root     root            4 Apr  6 10:20 X0R -> null
crw-------   1 root     tty        5,   1 Apr  7 11:13 console
lrwxrwxrwx   1 root     root           11 Apr  6 10:29 core -> /proc/kcore
lrwxrwxrwx   1 root     root           13 Apr  6 10:18 fd -> /proc/self/fd
brw-r-----   1 root     disk       3,   0 Mar 25 23:28 hda
brw-r-----   1 root     disk       3,   1 Mar 25 23:28 hda1
brw-r-----   1 root     disk       3,   2 Mar 25 23:28 hda2
brw-r-----   1 root     disk       3,  20 Mar 25 23:28 hda20
brw-r-----   1 root     disk       3,   3 Mar 25 23:28 hda3
brw-r-----   1 root     disk       3,   4 Mar 25 23:28 hda4
brw-r-----   1 root     disk       3,   5 Mar 25 23:28 hda5
brw-r-----   1 root     disk       3,   6 Mar 25 23:28 hda6
brw-r-----   1 root     disk       3,   7 Mar 25 23:28 hda7
brw-r-----   1 root     disk       3,   8 Mar 25 23:28 hda8
brw-r-----   1 root     disk       3,   9 Mar 25 23:28 hda9
prw-------   1 root     root            0 Apr  6 10:53 initctl
crw-r-----   1 root     kmem       1,   2 Mar 25 23:28 kmem
lrwxrwxrwx   1 root     root           12 Apr  6 10:55 log -> /var/dev/log
crw-r-----   1 root     kmem       1,   1 Mar 25 23:28 mem
crw-r--r--   1 root     root       1,   3 Mar 25 23:28 null
crw-r--r--   1 root     root       5,   2 Apr  7 11:48 ptmx
drwxr-xr-x   2 root     root            0 Apr  7  2001 pts
brw-r-----   1 root     disk       1,   0 Mar 25 23:28 ram0
brw-r-----   1 root     disk       1,   1 May 15  1996 ram1
lrwxrwxrwx   1 root     root            4 Apr  6 10:18 stderr -> fd/2
lrwxrwxrwx   1 root     root            4 Apr  6 10:19 stdin -> fd/0
lrwxrwxrwx   1 root     root            4 Apr  6 10:19 stdout -> fd/1
lrwxrwxrwx   1 root     root            7 Apr  6 10:19 systty -> console
crw-r--r--   1 root     tty        5,   0 Apr  7 11:15 tty
lrwxrwxrwx   1 root     root           13 Apr  6 10:08 tty1 -> /var/dev/tty1
lrwxrwxrwx   1 root     root           13 Apr  6 10:51 tty2 -> /var/dev/tty2
lrwxrwxrwx   1 root     root           15 Apr  6 10:09 ttyS14 -> /var/dev/ttyS14
crw-r--r--   1 root     root       1,   5 Mar 25 23:28 zero
Notes: I have hda1 to 9, but I only have 3 partitions, hda1 to 3 would have been OK. I'm using the /dev/pts file system, so I have ptmx and the pts mount point.ttyS14 is the device used by the Lucent modem. At the moment only ram0 is used.

/bin

-rwxr-xr-x   1 root     root       477692 Mar 25 22:32 bash
-rwxr-xr-x   1 root     root        10332 Mar 25 23:58 cat
-rwxr-xr-x   1 root     root        11540 Mar 26 09:42 chmod
-rwxr-xr-x   1 root     root        27188 Mar 25 22:46 cp
-rwxr-xr-x   1 root     root        12436 Mar 25 23:59 cut
-rwxr-xr-x   1 root     root        20492 Mar 25 22:33 df
-rwxr-xr-x   1 root     root         8760 Mar 26 00:18 hostname
-rwxr-xr-x   1 root     root        13904 Mar 25 22:45 ln
-rwxr-xr-x   1 root     root        47880 Mar 25 22:35 login
-rwxr-xr-x   1 root     root        47876 Mar 25 22:33 ls
-rwxr-xr-x   1 root     root        11356 Mar 26 09:49 mkdir
-rwxr-xr-x   1 root     root        10108 Apr  5 12:38 mknod
-r-sr-xr-x   1 root     root        14772 Mar 26 09:58 ping
-rwxr-xr-x   1 root     root        58140 Mar 26 09:55 ps
-rwxr-xr-x   1 root     root        19340 Mar 25 23:58 rm
-rwxr-xr-x   1 root     root        21012 Mar 26 00:18 setterm
lrwxrwxrwx   1 root     root            4 Apr  5 11:53 sh -> bash
-rwxr-xr-x   1 root     root         5528 Mar 26 09:42 sync
-rwxr-xr-x   1 root     root        93188 Mar 26 09:53 telnet
-rwxr-xr-x   1 root     root         6876 Mar 25 23:58 uname
Notes: I decided to use bash out of meer convenience (a command like history is handy). Busybox could have easily been used here instead.

/etc

-rw-r--r--   1 root     root            5 Mar 26 00:19 HOSTNAME
-rw-r--r--   1 root     root           28 Mar 26 00:00 adjtime
-rw-r--r--   1 root     root            0 Mar 26 00:23 fastboot
-rw-r--r--   1 root     root          226 Apr  5 11:55 fstab
-rw-r--r--   1 root     root          316 Mar 25 22:36 group
-rw-r--r--   1 root     root          745 Apr  6 10:31 hosts
-rw-r--r--   1 root     root          318 Mar 26 09:26 hosts.allow
-rw-r--r--   1 root     root          124 Mar 26 09:21 inetd.conf
-rw-r--r--   1 root     root         2781 Mar 25 22:34 inittab
-rw-------   1 root     root           60 Apr  5 11:58 ioctl.save
-rw-r--r--   1 root     root           27 Apr  5 11:58 issue
-rw-r--r--   1 root     root         3276 Apr  5 11:58 ld.so.cache
-rw-r--r--   1 root     root          223 Mar 25 23:49 lilo.conf
lrwxrwxrwx   1 root     root           22 Apr  5 11:53 localtime -> /usr/share/z
oneinfo/NZ
-rw-r--r--   1 root     root        10214 Mar 25 22:43 login.defs
-rw-r--r--   1 root     root           98 Mar 26 10:48 minirc.dfl
-rw-r--r--   1 root     root           14 Apr  5 11:58 motd
-rw-r--r--   1 root     root           24 Apr  5 11:59 mtab
-rw-r--r--   1 root     root          232 Mar 25 22:43 networks
-rw-r--r--   1 root     root         1108 Mar 25 22:37 nsswitch.conf
-rw-r--r--   1 root     root           28 Mar 26 15:58 passwd
drwxr-xr-x   2 root     root         1024 Mar 26 16:02 ppp
-rw-r--r--   1 root     root         1101 Mar 26 00:02 profile
-rw-r--r--   1 root     root          595 Mar 25 22:43 protocols
drwxr-xr-x   2 root     root         1024 Apr  5 12:02 rc.d
-rw-r--r--   1 root     root           92 Apr  5 17:29 resolv.conf
-rw-r--r--   1 root     root         5924 Mar 26 09:43 services
-rw-r--r--   1 root     root           52 Apr  5 12:14 shadow
-rw-------   1 root     root           52 Mar 26 15:58 shadow-
-rw-r-----   1 root     root          619 Mar 26 09:30 syslog.conf
-rw-r--r--   1 root     root         7881 Mar 25 23:04 termcap
Notes: The contents of many of these config files is shown later in this document.

/etc/rc.d

-rwxr-xr-x   1 root     root          374 Mar 26 09:55 rc.6
-rwxr-xr-x   1 root     root         1316 Apr  6 10:34 rc.M
-rwxr-xr-x   1 root     root         1199 Apr  6 10:32 rc.S
-rwxr-xr-x   1 root     root          360 Mar 26 15:48 rc.local
-rwxr-xr-x   1 root     root          404 Mar 26 00:09 rc.modules

/etc/ppp

-rw-------   1 root     root          263 Apr  5 12:50 chap-secrets
-rw-------   1 root     root            0 Jan 12 08:55 connect-errors
-rwxr-xr-x   1 root     root         1209 Dec 14 14:38 ip-down
-rwxr-xr-x   1 root     root         1209 Dec 14 11:53 ip-down.OLD
-rwxr-xr-x   1 root     root         1995 Mar 16 10:55 ip-up
-rwxr-xr-x   1 root     root         1946 Dec 14 11:53 ip-up.OLD
-rw-------   1 root     root          548 Mar 26 16:02 options
-rw-------   1 root     root          704 Apr  5 12:49 options.demand
-rw-------   1 root     root          211 Apr  5 12:50 pap-secrets
-rw-------   1 root     root          127 Mar 25 17:30 pppscript

/lib

At the moment, /lib is simply a complete copy of all the libs under /lib on a standard Slackware 7.1 install. /lib/modules contains modules for the lan cards; 3c509 and rtl8139, the ppp modules, the ltmodem module and all the ipmasq helper modules.

/sbin

-rwxr-xr-x   1 root     root        13844 Mar 25 22:35 agetty
-rwxr-xr-x   1 root     root        39456 Mar 26 00:10 depmod
-rwxr-xr-x   1 root     root        75860 Apr  5 12:08 e2fsck
-rwxr-xr-x   1 root     root        12900 Apr  5 12:09 fsck
-rwxr-xr-x   1 root     root           36 Apr  5 12:09 fsck.ext2
-rwxr-xr-x   1 root     root        34952 Mar 25 23:58 hwclock
-rwxr-xr-x   1 root     root        31992 Mar 26 00:26 ifconfig
-rwxr-xr-x   1 root     root       339576 Mar 25 22:30 init
-rwxr-xr-x   1 root     root        72860 Mar 26 09:59 insmod
-rwxr-xr-x   1 root     root        38008 Mar 26 10:07 ipchains
-rwxr-xr-x   1 root     root         8644 Mar 26 09:54 killall5
-rwxr-xr-x   1 root     root       109628 Mar 26 09:49 ldconfig
-rwxr-xr-x   1 root     root        87924 Mar 25 23:02 lilo
-rwxr-xr-x   1 root     root        72860 Mar 26 00:10 lsmod
-rwxr-xr-x   1 root     root        19812 Apr  5 11:54 mke2fs
-rwxr-xr-x   1 root     root        72860 Mar 26 00:10 modprobe
-rwsr-xr-x   1 root     root        60912 Mar 25 22:34 mount
-rwxr-xr-x   1 root     root         7576 Mar 26 00:17 reboot
-rwxr-xr-x   1 root     root        31064 Mar 26 00:26 route
-rwxr-xr-x   1 root     root        14744 Mar 26 00:17 shutdown
-rwxr-xr-x   1 root     root        16416 Apr  5 12:06 sulogin
-rwxr-xr-x   1 root     root         6956 Mar 26 09:41 swapoff
-rwxr-xr-x   1 root     root         6956 Mar 26 00:05 swapon
-rwsr-xr-x   1 root     root        28588 Mar 25 22:34 umount

/usr/bin

-rws--x--x   1 root     root        29572 Mar 26 15:38 chfn
-rwxr-xr-x   1 root     root       303420 Mar 25 22:46 elvis
-rwx--s--x   1 root     root       164312 Mar 26 10:18 minicom
-rws--x--x   1 root     root        35620 Mar 25 22:36 passwd
-rwxr-xr-x   1 root     root         6584 Mar 26 09:25 sleep
lrwxrwxrwx   1 root     root            5 Apr  5 11:53 vi -> elvis


-rwxr-xr-x   1 root     root         2129 Mar 26 15:37 adduser
-rwxr-xr-x   1 root     root        16916 Mar 26 10:10 chat
-rwxr-xr-x   1 root     root        31276 Mar 26 09:53 in.telnetd
-rwxr-xr-x   1 root     root        21224 Mar 26 09:19 inetd
-rwxr-xr-x   1 root     root        20280 Mar 26 09:25 klogd
-rwxr-xr-x   1 root     root       144980 Mar 26 10:09 pppd
-rwxr-xr-x   1 root     root        27684 Mar 26 09:25 syslogd
-rwxr-xr-x   1 root     root        21744 Mar 26 09:22 tcpd
-rwxr-xr-x   1 root     root        50548 Mar 26 15:37 useradd
Notes: The useradd, adduser is left over from when I was trying to set up the root user password. tcpd is there to allow restricted telnet access.

Startup Files

inittab

id:3:initdefault:
si:S:sysinit:/etc/rc.d/rc.S
su:1S:wait:/etc/rc.d/rc.K
rc:2345:wait:/etc/rc.d/rc.M
ca::ctrlaltdel:/sbin/shutdown -t5 -rf now
l0:0:wait:/etc/rc.d/rc.0
l6:6:wait:/etc/rc.d/rc.6
c1:1235:respawn:/sbin/agetty 38400 tty1 linux
c2:1235:respawn:/sbin/agetty 38400 tty2 linux
Notes: The standard progression is to run rc.S then rc.M. rc.0 and rc.6 are actually the same script, soft linked together. i really should create a rc.K, as going to single user is really the only way of being able to remount the / filesystem read/write.

/etc/rc.d/rc.S

PATH=/sbin:/usr/sbin:/bin:/usr/bin

# enable swapping
/sbin/swapon -a

echo "junk" >/tagfile 2>/dev/null
if [ -f /tagfile ];then
   echo "Root is already mounted readwrite"
   rm -f /tagfile
else
   fsck -A -a
fi

echo -n "Make /var ... "
/sbin/mke2fs -m 0 /dev/ram0 >/dev/null 2>&1
if [ $? -eq 0 ];then
   echo "OK"
   mount -t ext2 /dev/ram0 /var
   chmod 777 /var
   mkdir /var/run /var/log /var/adm /var/tmp /var/lock /var/dev
   mknod /var/dev/tty1 c 4 1
   mknod /var/dev/tty2 c 4 2
   mknod /var/dev/ttyS14 c 62 78
else
   echo "FAIL"
fi
# mount file systems in fstab (and create an entry for /)
# but not NFS because TCP/IP is not yet configured
/sbin/mount -a -v -t nonfs

cat /dev/null > /var/run/utmp

echo "Set system time from hardware clock ... "
/sbin/hwclock --hctosys
[ $? -eq 0 ] && echo "OK" || echo "FAIL"

# This loads any kernel modules that are needed.  These might be required to
# use your CD-ROM drive, bus mouse, ethernet card, or other optional hardware.
if [ -x /etc/rc.d/rc.modules ]; then
  . /etc/rc.d/rc.modules
fi
Notes: I've compiled into the 2.2.16 kernel to use 512Kbyte ram disks rather than the default of 4096Kbyte. There is no point in wasting the small amount of ram available to the gateway. Creating a 512K file system takes a fraction of a second and all that needs to be done is to create the directory structure and devices.

/etc/rc.d/rc.M

#!/bin/sh
echo "Going multiuser..."

# Screen blanks after 15 minutes idle time.
/bin/setterm -blank 15

# Set the hostname.  This might not work correctly if TCP/IP is not
# compiled in the kernel.
/bin/hostname `cat /etc/HOSTNAME | cut -f1 -d .`

HOSTNAME="`cat /etc/HOSTNAME`"
/sbin/ifconfig lo 127.0.0.1
/sbin/route add -net 127.0.0.0 netmask 255.0.0.0 lo

/sbin/ifconfig eth0 10.0.0.5 broadcast 10.0.0.255 netmask 255.255.255.0
[ $? -eq 0 ] && echo "eth0 configured OK" || echo "eth0 FAIL"
/sbin/ifconfig eth1 10.0.1.5 broadcast 10.0.1.255 netmask 255.255.255.0
[ $? -eq 0 ] && echo "eth1 configured OK" || echo "eth1 FAIL"

echo "Activate IPv4 packet forwarding"
echo 1 > /proc/sys/net/ipv4/ip_forward

echo -n "Start syslogd ... "
/usr/sbin/syslogd -p /var/dev/log
[ $? -eq 0 ] && echo "OK" || echo "FAIL"
sleep 1
echo -n "Start klogd ... "
/usr/sbin/klogd -c 3
[ $? -eq 0 ] && echo "OK" || echo "FAIL"

echo -n "Start inetd ... "
/usr/sbin/inetd
[ $? -eq 0 ] && echo "OK" || echo "FAIL"


# Remove stale locks and junk files (must be done after mount -a!)
/bin/rm -f /var/lock/* /tmp/.X*lock /tmp/core 1> /dev/null 2> /dev/null

# Update all the shared library links automatically
#/sbin/ldconfig

# Start the local setup procedure.
if [ -x /etc/rc.d/rc.local ]; then
  . /etc/rc.d/rc.local
fi

# All done.
Notes: Here the network interfaces are set up (remember I have two lan cards), syslogd and klogd are started and the last part is to run rc.local

/etc/rc.d/rc.local

#!/bin/sh
#
# /etc/rc.d/rc.local:  Local system initialization script.
#
# Put any local setup commands in here:
echo "Starting ltmodem"
/sbin/insmod -f ltmodem
echo "Starting Skymedia"
/usr/local/sm200d/skymedia start
echo "Starting IP Masquerade and Firewall"
/usr/local/bin/set-ipchains
echo "Starting pppd dial on demand"
pppd file /etc/ppp/options.demand
Notes:Add the modem module, start the sm200d satellite driver, set some basic firewall rules and start pppd in demand dialing mode.

Apr 23,2001