This proposal follows exactly the sections in http://www.oreilly.com/oreilly/author/ch02.html

• What will the book be good for?
• The Market for the Book
• The Outline
• Schedule
• Tools
• Who are you?

Detailed Outline Index

What will the book be good for?

This book is targeted at:

• Programmers who are interested in getting things done quicker by learning a functional programming language.
• Existing Erlang programmers who want to use the OTP libraries.
• Programmers who want to build distributed fault-tolerant applications.

The emphasis will be on "getting things done" and we will try very hard to dispell any myths that functional programming is only for mathematicians".

We will show programs that are an order of magnitude shorter than Perl or Java and which are easier to read and write.

The theme will be "fun code that works" the domain will be distributed applications. We'll do a lot examples with simple peer/peer and client/server architectures.

We'll show how to use Erlang and the Erlang libraries to make fun and interesting applications. Things like distributed ICQ/IRC/Napster client/servers, web servers, web caches, wiki webs, peer/peer chat systems and the like.

You can run the wiki web to get an idea of what the completed application looks like.

The book is intended to be used together with the Open Source Erlang distribution (see www.erlang.org It will make heavy use of the OTP libraries (which are part of the open source distribution).

Erlang is a new (ish) concurrent programming language designed for programming soft real-time distributed fault-tolerant applications.

Erlang is also the foremost of a new generation of functional programming languages designed for real-world applications. Erlang has been used for a number of heavy-weight and commercially successful applications.

The Open-Source Erlang distribution (which was released in 1998) contains an extremely rich set of tools for simplifying the construction of fault-tolerant distributed system.

The Market for the Book

The market for this book is potentially enormous - since there are few good hands on books in the Prolog/ML/scheme/Haskell school this market segment is grossly underpopulated.

There is already a very enthustiastic Erlang community - this book will help them and help spread the language. Erlang is spreading rapidly (without a good book, and without any advertising budget) - with a good book it will spead even more rapidly.

The market for this book are primarily people interested in learning a functional programming language. Unfortunately functional programming is view with suspicion by many programmers who think that FP is all about Greek and requires a deep knowledge of mathematics Nothing could be further from the truth.

The authors have taught Erlang to hundreds of regular programmers (i.e. programmers skilled in C/Java/perl). Once converted these programmers usually love Erlang and would never go back to low level languages like Java or Perl.

We believe that declarative programming language offer incredible advantages over conventional languages - our work with Erlang proves this without doubt - we feel it is now time to introduce this to a wider audience.

In comp.lang.functional I often read posts from people who are keen to try out a new functional programming language (because they've heard that FP is cool) but who are then frightened off by a heavy dose of theory and a swarm of "monads loving purists" who frighten off newcomers.

The functional programming world is split into two schools (the *heavy* theory schools (Monads, lazy polymorphic strong typing etc.) and the "impure" get things done school (scheme, Erlang, ...).

The former school of programming is well represented in the "classic literature", but the latter is poorly represented. many people on the Erlang list and on comp.lang.functional have expressed the opinion that "What Erlang needs is a good O'Reilly book" (a la Perl book) - I quite agree (hence this proposal)

Aside: I have just attended the 7'th Erlang conference - which had over 130 attendees - They kept asking me - When are you going to write a new book.....

Erlang was designed to be "an easy to use functional programming language" it uses dynamic typing (a la smalltalk/scheme) and has a simple process based concurrency model.

Erlang was released as Open Source in 1998. Tens of thousands of downloads of the system have been made and there is a an increasingly enthusiastic user group and mailing list.

One primary obstacle to the widespread use of Erlang is "a good hands-on book". The proposed book is an attempt to solve this problem.

The first every book "Concurrent programming in Erlang" (Prentice hall) described the language - but not the libraries.

1. An Overview Of Erlang

1.1. Your first ever compiled Erlang program

To avoid plagiarism, we start with "Goodbye world":

-module(goodbye). -export([world/0]). world() -> io:format("Goodbye world\n").

Use you favorite editor to store this in a file. Call the file goodby.erl.

Now start the erlang system, by typing erl, compile the program and run it:

erl Erlang (BEAM) emulator version 5.0.1.1.b3 [source] [threads] Eshell V5.0.1.1.b3 (abort with ^G) 1> c(goodbye). {ok,goodbye} 2> goodbye:world(). Goodbye world ok

That's it you're up and running. The command erl starts the Erlang shell. c(goodby> compiled the module contained in the file goodbye.erl and goodby:world() evaluated the function world() in the module goodbye.

1.2. Writing your first Erlang script

If you want to run "goodby world" as a script, then create file called goodbye.sh containing the following:

#!/usr/bin/env escript main(_) -> io:format("Goodbye world\n").

This can be run as follows:

[joe@pippi] > chmod u+x goodbye.sh [joe@pippi] > ./goodbye.sh Goodbye world [joe@pippi] >

Running Erlang programs as shell scripts is not as efficient as compiling programs, but is often a lot more convenient.

1.3. Factorial

Now you'll want to progress to something more serious, like factorial -- this is a recursive (Don't panic) function. Just for fun we'll implement this as a shell script:

#!/usr/bin/env escript main([Arg1]) -> N = list_to_integer(Arg1), X = factorial(N), io:format("factorial ~s = ~w~n", [Arg1, X]). factorial(0) -> 1; factorial(N) -> N * factorial(N-1).

Running it is easy:

./factorial.sh 123 factorial 123 = 121463043670253296757662432418812958554542170884833823 1532891816182923589236216766883115696061264020217073583522129404778259 1091570411651472186029519906261646730733907419814952960000000000000000 000000000000

Although this example was very simple it has illustrated a number of points.

Writing a shell script

Parsing the command line arguments

This we did almost in passing. main([Arg1]) -> ... extracted The first of the command line arguments into the variable Arg1. All command line argumnets are strings, so the statement N = list_to_integer(Arg1) converted the input string to an integer N (strings are represented as linked lists in Erlang).

Writing a recursive function

The function factorial.

Bignum arithmetic

Also note in passing that all integer arithmetic is exact - thus the system could easily compute the exact value of factorial(123) and silently produce an incorrect value. This incidently, removes an entire class of programming errors, namely those cased by inadvertent integer overflows!

1.4. Pattern matching

2. Sequential programming

2.1. primitive data types

There are 4 primitive data types are atoms (examples monday, red), floats (examples 3.14159, 1.23e29 and integers (examples, 1234, 129872894696491649264961249169486).

2.2. lists and tuples

There are two compound data types. tuples written {T1,T2,..,Tn} and lists written [T1,T2,..,Tn] where T1, T2, .. Tn are either primitive or compound data types.

Tuples are like structs in C and are used for storing a fixed number of data items. Lists are like linked lists in C and are used for storing a variable number of data items.

2.3. lists (the library)

Examples from the list library.

2.4. map, filter, split

A few simple examples of some of the higher order functions in the lists library.

We start off with higher order list programming (paradoxically this is very easy to understand) you can understand this even if you don't know what a list is. We do this though dialogues with the shell.

Here's an example dialogue with the shell:

1> L=[1,2,3,4]. [1,2,3,4] 2> lists:map(fun(X) -> 2*X end, L). [2,4,6,8] 3> lists:map(fun(X) -> X*X end, L). [1,4,9,16]

This is easy to explain and understand.

• Line 1 builds a list L containing four integers 1,2,3 and 4.
• Line 2 maps the function fun(X) -> 2*X end over the list, i.e. doubles each element in the list.
• Line 3 squares each element etc.

Having given the examples, we can then move readily explain the syntax of lists, what they "mean" etc.

Moving on from lists, we introduce tuples and functions over tuples:

1> L=[{10, joe},{5,jane},{60,mark}]. [{10,joe},{5,jane},{60,mark}] 2> lists:sort(L). [{5,jane},{10,joe},{60,mark}] 3> lists:map(fun(X) -> element(1,X) end, L). [10,5,60] 4> lists:reverse(lists:map(fun(X) -> element(1,X) end, L)). [60,5,10]

2.5. list comprehensions

List comprehensions are wonderful. To get an idea let's try to compute all Pythagorean triangles, having a sides of total length less than N:

#!/usr/bin/env escript -mode(compile). -export([main/1]). main([Arg1]) -> N = list_to_integer(Arg1), X = pythag(N), io:format("pythag(~w) = ~p~n",[N, X]). %% Return the Pythagorean triangles with sides %% of total length less than N pythag(N) -> [ {A,B,C} || A <- lists:seq(1,N), B <- lists:seq(1,N), C <- lists:seq(1,N), A+B+C =< N, A*A+B*B == C*C ].

We can test this as follows:

[joe@pippi] > ./pythag.sh 35 pythag(30) = [{3,4,5},{4,3,5},{5,12,13},{6,8,10},{8,6,10},{12,5,13}]

The tuple {12,5,13} represents a Pythagorean (right angled) triangle since 12^2 + 5^2 = 13^2 wait a moment I can check this in the Erlang shell

[joe@pippi] > erl 1> 12*12+5*5==13*13. true

and 12+5+13 = 30 which is less than 35.

The notation [X || Y ] means construct the set of X such that Y is true. The symbol || is pronounced such that. P <- Q is read P is taken from Q. In English we can read the program as follows:

[{A,B,C}

Construct the set of tuples {A,B,C}

||

such that

A <- lists:seq(1,N)

A is taken from the list lists:seq(1,N) - Note if N is (say) 10 then lists:seq(10) just means the sequence [1,2,3,4,5,6,7,8,9,10], in which case A will successively be bound to 1 then 2 etc. and finally 10.

B <- .. C <- ..

Same as for A

A+B+C =<< N

A+B+C is less than or equal to N,

, comma

and,

A*A+B*B == C*C

A squared plus B squared equals C squared.

Then we'll do list comprehensions, leading up to quick-sort. Again easy:

sort([]) -> []; sort([H|T]) -> sort([X||X<-T, X < H]) ++ [H] ++ sort([X||X<-T,X>=H]).

This is rather nice, quick-sort in two statements. Simple, understandable, efficient.

This can be read declaratively like this:

• sort([]) -> []; The sort of an empty list is the empty lists.
• sort([H|T]) -> sort([X||X<-T, X < H] ++ [H] ++ [X||X<-T,X>=H] To sort a list whose first element is H the rest of list being T then find all elements less than H (The notation [X||X<-L, X<H] means "the list of elements X such that X is taken from L and X is less than H) sort them, append them to H and append this to the sort of all elements greater than H"

2.6. bignums

Now a little diversion with bignums.

17> X = 12937186381648168628149624896129846891264982346928146. 12937186381648168628149624896129846891264982346928146 18> Y=83468264826486248628461982349128648912634981264981264981234. 83468264826486248628461982349128648912634981264981264981234 19> X*Y. 107984449901302073454653146731258818648233194910204543873\ 4559134295011016190913368983651698805436332462236412164

Followed by an explanation. The purpose is twofold. Firstly we get the idea that bignums are more convenient to use and think about than fixnums (think 32 bit) - this fact usually impresses people :-) secondly, and more importantly, that use of bignums eliminates a large class of programming errors (integer overflow).

2.7. functions and clauses

now, that the ground is repaired, we show how to write functions over lists and tuples, we've already used map and sort and reverse etc. So now we show how to implement them as recursion over lists.

map is easy:

map(F, []) -> []; map(F, [H|T]) -> [F(H)|map(F, T)].

Or it will be when we have finished with it.

2.8. modules, imports and exports

2.9. funs

2.10. records

2.11. catch/throw

2.12. macros

Finally we run through macros and records.

At the end of this chapter the reader should have a good idea as to how to write sequential functions.

3. Sequential libraries and patterns

We go through the major libraries. Lots of examples of the most useful and most commonly used function.

3.1. lists

3.2. file

3.3. code

3.4. map/filter/splitwith

3.5. regexp

3.6. ets

3.7. dets

The purpose of this chapter is to illustrate the major libraries, and how they are used.

The format is first examples in the shell and then summarized manual pages.

For example, here are some simple and very commonly used list processing functions:

1> X = [1,2,3,4,5,6]. [1,2,3,4,5,6] 2> Y = lists:reverse(X). [6,5,4,3,2,1] 3> X ++ Y. [1,2,3,4,5,6,6,5,4,3,2,1] 4> lists:sort(X++Y). [1,1,2,2,3,3,4,4,5,5,6,6] 5> lists:nth(4, X). 4

• Line 1 creates a variable X contains the list [1,2,3,4,5,6].
• Line 2 reverses the order of the elements in the list, storing the results in the variable Y..
• Line 3 - appends the lists X and Y. The infix append operator is written ++.
• ...

4. Real sequential applications

4.1. Yecc/Leex

From Joe examples.

4.2. find

I have a program for this.

4.3. Rsync

Nice program

4.4. remove_dups

Uses find.

4.5. rsa

My examples,

4.6. MD5 authentication

I have the code for this.

5. Interlude - errors

5.1. Theory of error handling

The Erlang way of handling errors is one of the most misunderstood points of the language. The Erlang principle

• LAW: Ensure clean separation of Error handling and "normal case code"

Don't mess up beautiful code which handles normal cases with messy code for error handling.

Suppose deep down in some Erlang program we require a function which converts the atom a to the integer 1, and the atom b to the integer 2. Call this function tr. Obviously we can define tr/1 thus:

tr(a) -> 1; tr(b) -> 2.

So far so good -- now what about errors? - The Erlang way of doing things is to totally separate the code for error handling from the place were the error occurred

+----------------+ +-----------------+ | | | Error handling | | tr(a) -> 1; | - - - - link - - - - | process | | tr(b) -> 2. | +-----------------+ | | +----------------+ If an error It is fixed here occurs here

In the application program makes a mistake, for example calling tr(c) -- the application process will fail and the error handling process IF WRITTEN will correct the error.

Note, you don't always need to write an error handling process. You only need an error handling process if you want to try to correct the error at run-time. If you don't write an error handling process then the process were the error occurred will crash and a message sent to the system "error_logger" process. The error logger reports the error on standard IO (though this behavior can be changed).

What do we find in practice? Often people write this type of code:

tr(a) -> 1; tr(b) -> 2; tr(Other) -> do something ...

Oh dear! -- Firstly this code has transformed a CORRECT PROGRAM into an INCORRECT PROGRAM. After all in our original specification tr(c) was undefined -- in the modified program tr(c) HAS a value (i.e. whatever the code in "...do something ..." returns).

So what should you do when the error occurs? When you read code that handles exceptional cases like this you will find that the _writer_ of the program doesn't usually know what to do either. One often finds code like this:

tr(a) -> 1; tr(b) -> 2; tr(Other) -> io:format('map called with wrong value:~w\n', [Other]).

Is that any better? - Yes and No. We get a helpful diagnostic BUT tr(c) evaluates to 'ok'.

This will most probably cause the program to crash at some other point in time.

The original version:

tr(a) -> 1; tr(b) -> 2.

Was actually much better. Since:

• It is TOTALLY CORRECT
• You get a helpful????? (I know - we're working on it) diagnostic on standard output if it is called with an incorrect argument
• The process dies as soon as the error occurs (so their are no knock on effects and it is easy to find the error)

tr(a) -> 1; tr(b) -> 2; tr(Other) -> io:format('map called with wrong value:~w\n', [Other]), exit(i_die).

Other attempts to fix errors use "catch" and "throw" - now we put the emphasis on error handling on the caller:

case catch tr(X) of {'EXIT', Why} -> ... do something ...; NormalCase -> .... end

Now we have the same problem as before only its been pushed to a different part in the code. What do we write in the "... do something ..." part of the code?

Something is seriously wrong here, we can't continue. Write an error diagnostic and crash BUT THIS IS THE DEFAULT BEHAVIOR WITHOUT writing any code at all!

Principles of Error handling

• Crash as soon as possible after detecting an error - do not make life more difficult for yourself by trying to carry on. This has to so with detection of errors.
• try to give a helpful diagnostic when you crash.
• Isolate hardware with a device driver. Do the conversions in one place.
• Don't try to carry on if you get a programming error.
• Check values only after human interactive input. Thereafter do not check values for "reasonableness".
• Trust other peoples code let it crash if it goes wrong!
• Do things in a generic manner.
• Program the "normal case" let the system take care of the errors
• Get different people to write the error handling and normal case code.

From earlier writing.

5.2. How to code

From programming rules

6. Concurrent programming

Rewrite from scratch.

6.1. processes

6.2. links

6.3. Distribution

6.4. Sockets (gen_tcp)

6.5. epmd

6.6. promises

7. Concurrent libraries and patterns

7.1. gen_servers

From OTP doc - re-written

7.2. What more?

8. Real Concurrent applications

In this section I'd like to give a complete example of a fault-tolerant server. I imagine a simple name server that associates a fixed name with a dynamic resource.

8.1. The name server

8.2. The client code

8.3. Making the server into a demon process

This note describes how to make an Erlang daemon.

A daemon is a program that runs all the time in the background on your computer. It is started when the machine boots and is stopped when the system is stopped.

8.3.1.  FreeBSD

I use the two scripts wiki-server.sh which must be placed in /usr/local/etc/rc.d/ and wiki.sh.

When the system starts all the scripts in /usr/local/etc/rc.d/ are executed with argument start. Here is wiki-server.sh. wiki-server.sh

/home/wiki/wiki.sh runs as the user wiki and contains the following: wiki.sh

Evaluating wiki.sh start starts the wiki daemon. The script is simple. The only points to notice are:

• Use an explicit path to erl (i.e. ERL=...)
• Erlang is started with flags -heart -detached - the -heart flag starts a heartbeat process. If Erlang dies then the heartbeat process will detect this and restart Erlang with the command specified in the HEART_COMMAND environment variable. -detached means run without a terminal.

RewriteEngine On RewriteRule ^/wiki/(.*)$ http://127.0.0.1:5999/wiki/$1 [L,P] %% Requests directed to www.bluetail.com/wiki %% will be redirected to localhost:5999/wiki Thanks to Petru Paler petru@ppetru.net for this tip.

8.4. Starting the demon

8.5. socket authentication with MD5

8.6. eol (erlang on line()

8.7. peer-to-peer

8.8. client server

9. Interlude - systems programming

This is the "theory" of the OTP stuff.

10. The OTP application structure (theory)

This is similar to otp design principles

10.1. gen server

10.2. supervision

10.3. logger

10.4. FSM

11. A real word OTP application

A fault-tolerant wiki web.

12. State of the art

This is for stuff that (at the time of going to press hasn't yet made it into the system). These are source forge project or similar.

12.1. SAE

12.2. egtk

13. Appendix 1 - Obtaining the system

14. Appendix 2 - Installing the system

15. Appendix 3 - Major Libraries functions

16. Appendix 4 - Quick ref to building an OTP application

17. Appendix 5 - Nitty gritty

17.1. Making an Erlang daemon on Linux or free BSD

17.2. Windows demons

17.3. Remote shell debugging

17.4. Windows demons

17.5. Erlang shell editing commands

17.6. Debugger

17.7. Cookies

17.8. Compiling

17.9. rigging epmd

17.10. Tracing

18. Appendix 6 - Erlang description

19. Appendix 7 - Erlang manual page

• A statement of purpose for each chapter.

• A fine-grained breakdown of the contents of each chapter (but see below).

• An explanation of the rationale for the overall organization of the book.

• An indication of what use you will make of illustrations.

• Any special considerations that apply--unusual format, use of color, hard-to-get illustrations, or anything else calling for unusual resources.

• An estimated page count for the book.

Schedule

Tools

I'd like to write this in EHTML.