Stand Alone Erlang - version 3.0

• The basic kit
• How to make and distribute a stand-alone application
• Initialization
• How basic loading works
• How demand code-loading
• Building SAE
• Download
• Ecc and elink
• Example
• Performance
• To do

This is version 3.0 of the support routines for "SAE" (Stand Alone Erlang) - with this you can make small stand-alone Erlang applications which run quickly from the Unix command line prompt (see below for measurements).

With SAE you can distribute complete applications in a very small number of files.

SAE is currently work in progress and is not supported. This document describes how to put together SAE for the Linux version of otp_src_P8A-20011007-SNAPSHOT.

Have fun,

/Joe (joe.armstrong+nospam@telia.com)

Basic Kit

The basic Erlang run-time kit now requires seven files and a total of 3.5 Mbytes of storage, these files are:

To build these files follow these instructions.

beam_evm

is the compiled Erlang emulator.

.lib files

are packed library archives containing compressed BEAM code and include files.

ecc

is the erlang compiler

elink

is the erlang linker

These seven files contain the logical equivalent of about 500 files in the distributed system. Having one .lib file per library makes system administration a lot easier :-)

How to make and distribute a stand-alone application

Firstly you need to distribute the 7 files mentioned in the introduction.

If you have a small application, composed of a few files, then you can build the application as follows:

   > ecc F1.erl F2.erl ....
   > elink -o Exec -m F1 F2 F3 ... -s Mod Func -args ...

If you have several inter-related applications which use a common set of library routines then you can build a private library, using ml_lib:pack/3 - for example, evaluating:

    mk_lib:pack_library("joe.lib", "/home/joe/src/apps/src",
	                           "/home/joe/src/apps/include").

makes a private library called joe.lib from all the BEAM code in /home/joe/src/apps/src and all the .hrl files in /home/joe/src/apps/include.

This library should be installed in the same directory as the directory containing stdlib.lib

Initialization

All .lib files must be placed in the library directory The library directory is defined to be the first directory found which contains the file stdlib.lib.

The following directories are searched:

• First the current directory.
• Then the value of environment variable ERL_SAE_LIB
• Then /usr/local/lib/erlsae/
• Finally ../dist

This is done by the function demand_prim:local_lib_dir/0.

The motivation for this search order is as follows:

1. Following a "default" installation, the system will be installed in "a well-know place" (here /usr/local/lib/erlsae/) and the executable beam_evm will be stored somewhere on the user's path (probably /usr/local/bin).
2. If the user cannot install the system in the standard place they can install it privately, and set the environment variable ERL_SAE_LIB to point to the installation directory.
3. If the user cannot set an environment variable, they can run the system from the current directory.
4. The final case is so that the system can find the initial bootstrap.

The demand code loader will only demand load code from the library directory.

How basic loading works

The SAE distribution consists of one large executable (beam_evm) this is the "Erlang engine".

All stand-along executable have the following format:

   #!/usr/bin/env beam_evm
   :NNNNN
   <<binary data>>

The first line #!/usr/bin/env beam_evm locates the Erlang executable.

The second line :NNNNN is an integer representing the size of the binary data <<binary data>> which follows.

When beam_evm starts it gets passed the file name of the executable, it then reads NNNNN and the binary data which follows. The binary data is an Erlang binary (created by the BIF term_to_binary) - beam_evm then spawns a parallel process which evaluates ring0.start(Env, Args) - where Env is the binary which was contained in the executable file and Args comes from the command in arguments.

The version of sae-3.0/src/ring0.erl should replace the version found in the distribution at otp_src_P8A-20011007-SNAPSHOT/erts/sae/src/ring0.erl

ring0.erl is pre-loaded - in fact it is the only Erlang code that needs to be pre-loaded.

Here is ring0.erl.

-module(ring0). %% Purpose : Start up of Erlang system. -export([start/2]). start(Env, Argv0) -> {Ms,{M,F,Args}} = binary_to_term(Env), Loaded = load(Ms, []), Argv = [binary_to_list(B) || B <- Argv0], run(M,F,Args,Argv). run(M, F, A0, A) -> case erlang:function_exported(M, F, 2) of false -> erlang:display({fatal,error,module,M,'does not export',F,'/2'}), halt(1); true -> apply(M, F, [A0, A]) end. load([{Mod,Code}|T], Loaded) -> case erlang:load_module(Mod, Code) of {module,Mod} -> load(T, [Mod|Loaded]); Other -> erlang:display({bad_module,Mod}), erlang:halt(-1) end; load([], L) -> L.

The executable is created by the program elink.erl, the following lines of code in taken from elink.erl are used to create the start term Ring0term - this term must agree with the term structure used in ring0.erl.

... make1(OutObj, Ms, StartMod, StartFun, Env, Demand, Arg1) -> check_start_function(StartMod, StartFun, Ms), %% The following line must agree with the code in ring0.erl Ring0Term = case Demand of true -> {Ms, {demand_prim, startMeUp, {StartMod, StartFun, Arg1}}}; false -> {Ms, {StartMod, StartFun, Arg1}} end, make_program(OutObj, Ring0Term, Env). %% packs Ms = [{Mod,Bin}], StartMod, StartFun, StartArgs make_program(OutObj, BLM, Env) -> Payload = term_to_binary(BLM), Len = size(Payload), %% erlang:display({joe,elink,size,Len}), Header = ["#!/usr/bin/env beam_evm\n", Env, ":", integer_to_list(Len),"\n"], Everything = [Header,Payload,"--end--\n"], file_prim:write_file(OutObj, Everything), file_prim:change_mode(OutObj, 8#755), true. ...

How demand code loading works

Demand code loading works as follows:

• First we pack all the BEAM code into stdlib into stdlib.lib, This is done by evaluating mk_libs:pack_library(stdlib) ... etc.

  Then we pack all the code in kernel and compiler.

  .lib files are simple disk archives maintained by the module pds (primitive disk store) - PDS use no library routines so it can be used to load code itself.

• A new version of code (in the file fake_demand_code.erl is used which loads code from the .lib archives instead of from the standard places.

Note that all include files (Lib/include/*.hrl) are also added to the .lib files. These are needed for compiling modules which contain:

   -include_lib("kernel/include/file.hrl").

directives.

A hacked version of epp (called epp_hacked) is used which correctly retrieves the appropriate include file from the .lib archives as necessary.

Download

• sae-3x0.tgz - this is an add- on to the open source erlang source code release otp_src_P8A-20011007-SNAPSHOT.

Building SAE

These are the instruction for building SAE on my home machine in the directory /home/joe.

1) Fetch the latest system http://www.erlang.org/download/otp_src_R8B-0.tar.gz

2) Unpack the distribution in you home directory. this creates the top-level directory /home/joe/otp_src_P8A-20011007-SNAPSHOT. Configure and make.

   > cd /home/joe
   > gunzip zcat otp_src_R8B-0.tar.gz | tar -xvf - 
   > cd otp_src_R8B-20011015-SNAPSHOT 
   > ./configure
   > make
   %% drink a few cups of coffee chat to mates etc. - wish I had
   %% a 4GHz machine ....

3) Set a path to locate the correct versions of erl and erlc:

> export PATH=/home/joe/otp_src_R8B-20011015-SNAPSHOT/bin:$PATH 

4) Unpack sae-30.tgz and copy the version of ring0.erl in the package to the distribution.

This replaces the version of ring0 in the distribution with a better one :-).

> cp ring0.erl /home/joe/otp_src_R8B-20011015-SNAPSHOT/erts/sae/src/ring0.erl

5) Build sae.

   > export ERL_TOP=/home/joe/otp_src_R8B-20011015-SNAPSHOT   
   > cd $ERL_TOP/erts
   > make sae

6) Copy the virtual machine to somewhere in your path:

   > cp $ERL_TOP/bin/i686-pc-linux-gnu/beam_evm  ~/bin

6) Strip it

   > cd ~/bin
   > strip beam_evm

This reduces the size of beam_evm from 1785363 bytes to 624504 bytes.

7) Build the support tools

   > cd sae-3.0/src
   > make

This will build the libraries into sae-3.0/dist.

8) set an environment variable so you can find the libraries.

   > export ERL_SAE_LIB=/home/joe/sae-3.0/dist"

9) Test it.

  > cd sae-3.0/examples
  > make
  > ./test_demand 40
  Link args=["1","2","3"]
  factorial 40=815915283247897734345611269596115894272000000000   

ecc and elink

ecc foo.erl bar.erl ....

compiles foo.erl, bar.erl, ...

elink [-d] -o Out -m Mod1 Mod2 Mod3 .. -args A1 A2 A3 .. -s Mod Func

Links a number of modules to make an executable.

-o Out specifies the name of the executable file.

-m Mod1 Mod2 specifies the modules to be used, these modules must have been compiled.

-args A1 A2 A3 .. defines a number of "link time arguments".

-s Mod Func specifies a "start function" to be called when the application is started.

When the application is started Mod:Func(Args0, Args1) will be evaluated, where Args0 contains a list of the "link time arguments" A1, A2, ..and Args1 contains a list of the command line arguments.

Example

Here is a complete example which is taken from the examples directory in the distribution.

-module(test_demand). -export([main/2]). main(A0, Args) -> case Args of [_,N|_] -> case (catch list_to_integer(N)) of {'EXIT', _} -> usage(), exit({not_an_integer, N}); Int -> Fac = fac(Int), io:format("factorial ~p=~p~n",[Int, Fac]) end; _ -> usage() end, erlang:halt(). usage() -> io:format("Usage test_demand NNNN~n"). fac(0) -> 1; fac(N) -> N * fac(N-1).

To compile and link we give the following commands:

  > ecc test_demand.erl
  > elink -E -o test_demand -m test_demand -s test_demand main

Performance

Here are a few figures for hello.erl (hello world) which is in the examples directory.

 
> time ../dist/ecc hello.erl
"hello.erl" {ok,hello}
0.70user 0.10system 0:00.79 100%CPU 

> time ../dist/elink -d -o hello -m hello -s hello main
0.11user 0.02system 0:00.14elapsed 92%CPU

> time ./hello
'hello world'
0.01user 0.01system 0:00.02elapsed 100%CPU

> ls -l hello
-rwxr-xr-x    1 joe      staff         396 Oct 17 14:59 hello  

The next test is for ecat.erl (See the examples directory for code) - this is a mini-version of the Unix "cat" utility.

 
> time ./ecat <../dist/stdlib.lib >aa
0.02user 0.05system 0:00.07elapsed 95%CPU
> wc ../dist/stdlib.lib ecat
   4770   26689 1124078 ../dist/stdlib.lib
     18     106    4429 ecat
   4788   26795 1128507 total> ls -l ../dist/stdlib.lib

This is fast enough (0.07 sec, on my 433MHz Celeron ) and small enough (4429 bytes) to be useful for shell scripts, cgi stuff etc.

To do

The use of .lib files makes it possible to write a "library manager" (somewhat like the red-hat package manager).

I am working on an extension to Erlang, whereby modules would begin with a lib declaration. The standard module lists, would begin:

   -module(lists).
   -lib(stdlib).
   ...

Meaning that lists belongs to stdlib (naturally enough).

With proper use of a lib system we could require that "module names are unique to within a library name but not globaly unique" - which provides a partically solution to the "module namespace pollution problem".

This facility is planned in a future release of sae.

It would also be very nice to see this stuff ported into windows - volunteers ...