Previous | Next

Objectives| Introduction| The Basics of Shell Programming| Shell Comments| Using Shell Variables| Predefined Shell Variables| Repeated Action Commands| Some Other Useful Commands| Conclusion| Review Questions

Section 3


SHELL PROGRAMMING


Objectives


At the end of this section you will

Introduction


Shell programming is a basic skill that every System Administrator should have. The Systems Administrator must be able to read and write shell programs because

The Basics of Shell Programming


Simply put, a shell program (sometimes called a shell script or a shell procedure) is a text file that contains standard UNIX commands and maybe some specific shell commands. Each command must be on its own line. A shell program is created usually to perform some repetitious job that requires too much typing to be typed in everytime the task must be done.

A shell program is executed by the appropriate shell reading the commands in the program one at a time and executing the commands almost exactly as if you were typing them in one at a time.

For example:

The following command will display the number of files and directories in the current directory
ls | wc -w
Not much typing but if you were going to want to use it many times you might want to cut down on the number of keystrokes. The simplest way would be to create a shell program that contains this command.


How to Create a Shell Program


  1. Choose which shell to use.
    Most Systems Administrators will usually write shell programs using the Bourne shell as this is the only shell guaranteed to be on all UNIX systems.

    All shell programs in this course will use the syntax of the Bourne shell.

  2. Create a text file containing the commands.
    Using your favourite text editor create a file containing all the commands (one to a line) you want in the shell program. The first line of the file is used to indicate the name of the shell to use when executing your shell script. This is signified by a line starting with #! followed by the path of the shell you wish to use. (Table 2.4 lists some of the names of shells).

    For example:

    	#!/bin/sh		for the Bourne shell
    	#!/bin/csh		for the C shell
    
  3. Make the shell program executable and readable.
    Using the chmod command from Section give the appropriate users execute and read privilege on the file. (The shell has to be able to read the file to execute it.)
    The following command should perform the required task (the chmod command will be explained in a later section). chmod u+rx filename

  4. Execute the shell program.
    Execute the program by typing the name of the file.
A shell script can also be executed by executing the shell with the shell script as a parameter. This method makes obvious the way in which shell scripts are executed. The shell takes the script as input, executing each command in turn.

For example:

To execute a shell script called username with the Bourne shell
sh username
To execute the shell script username2 with the C shell
csh username2
Exercise 3-1. Create a shell procedure called fc (file count) that uses the command ls | wc -w to count the number of files in a directory.

Exercise 3-2. Create a shell procedure called d that displays the message The current directory is /home/temp and it contains and on the following line displays the output of the ls command on the current directory.


Shell Support for Programming


To be truly useful a programming language must provide the following services Conditional and repeated action commands are some of the extra commands that each shell recognises and are not UNIX commands. Different shells have different syntax for the two types of commands. All the examples of shell programming in this section will use the syntax of the Bourne shell.
How do you tell a UNIX command from a Shell command?

A UNIX command will always have an executable program stored as a file somewhere on the disk. Try the command which commandname. This command searches through the current path for an executable program called commandname.

The code to perform a shell command is built into the shell program and won't be stored in a file.

The Bourne shell is used by most administrator's because it is the one shell guaranteed to be on all UNIX machines.


Shell Comments


Any good programmer can tell you the reasons why comments are important. It is especially true for UNIX shell procedures that can be truly mystifying, even to the expert.

Comments are especially important when any shell programs you write as a System Administrator are likely to have to be understood and maintained by a fellow administrator in a few years time.

Shell programs use the # character to signify comments. Anything after a # character until the end of the line is considered a comment and is ignored by the shell.

As an administrator you should choose a format for all the shell programs you and anyone else on the administration team will write.

For example:

		# NAME:	test_shell
		# AUTHOR:	David Jones
		# PURPOSE:	Demonstration of comment style
		# HISTORY:	19/05/94	Created
		#		25/05/94	Modified to show mods

		ls | more

Using Shell Variables


Like any language the shell has to provide a variable mechanism that can be used to access, store and modify values.

Most programming languages have rules that restrict the format of variable names. For the Bourne shell, variable names must start with either a letter or an underscore character. It can then be followed by zero or more letters, numbers or underscores

To assign a value to a variable you use the following format variable=value
(Remember this is the Bourne shell syntax. Using the C shell the format would be set variable=value)

No spaces around the equals sign. If you want to assign a string that contains spaces you must surround it with double quotes
For example:
count=1
prompt="hello there my friend"
bin_dir=/usr/bin
If you wish to use a shell variable's value you must precede the name of the variable by a dollar sign. The shell will examine the command line and see that there is a dollar sign. The shell recognises the $ as a special character that signifies a shell variable name. It will then replace the shell variable name with its value. Finally it will execute the command. (This process is discussed in section 2.)

For example:

echo $prompt
cd $bin_dir
A shell variable does not have to be used for data only. It can be used for commands.

For example:

command=mv
old_location=/home/david/temp.dat
new_location=/usr/adm/account
$command $old_location $new_location

Using {}


In some cases you will wish to use the value of a shell variable as part of a larger word. Curly braces {} are used to separate the variable name from the rest of the word

For example:

The user wants to copy the file /etc/passwd into the directory /home/david.
The following shell variables have been defined.
directory=/etc/
home=/home/david

First attempt

cp $directorypasswd $home
This won't work because the shell is looking for the shell variable called directorypasswd (there isn't one) instead of the variable directory.
Correct Method
cp ${directory}passwd $home
Surround just the variable name with curly braces, the shell will substitute in the variable's value first

Exercise 3-3 Which of the following are valid shell variable names.

	a)  temp	b)  temp.doc
	c)  1abc	d)  abc#1
	e)  abc_1_	f)  TeMpVaR
Exercise 3-4 Assuming the following command has already been executed.
home=/home/david/
Write commands using the home shell variable to do the following
a) list the contents of the directory /home/david
b) rename the file temp in the directory /home/david to temp2

Predefined Shell Variables


Not only does the shell allow the user to assign values to shell variables, it also supplies a number of predefined shell variables which are used for a variety of purposes including

Shell Variables and Environment Control


When you first log onto a UNIX machine the environment in which you are working is initialised. This environment includes things like For most of these objects there is an associated shell variable that holds the value. To examine the current state of your environment execute the shell command set.

Table 3.1. lists some of the shell variables typically set for a user's environment (there are numerous others).

Variable	Purpose
HOME your home or login directory PS1 PS2 your first and second command prompts SHELL your current shell UID your user id USER your username TERM the type of terminal you logged in on DISPLAY your X-Windows display (if X is being used) PATH Your execution path Table 3.1. Environment Shell Variables.
For example:
The variable PATH holds the list of directories (separated by :) in which the shell will search for executable programs when you enter a command.
$ echo $PATH
/usr/ucb:/usr/bin:/bin:/usr/local/bin
You can change this path simply be changing the value of the variable PATH
$ PATH="/bin"

or
$ PATH=`echo $PATH`:/home/jonesd/bin
add the directory /home/jonesd/bin onto the end of the existing path

Shell Variables and Parameter Passing


When you enter a command (be it an executable program or a shell program) you enter the command name and possibly then some arguments or parameters. The arguments or parameters generally effect the way in which the program works. The Bourne shell has a number of predefined shell variables that are used from within shell scripts to access these parameters.

For example:

Assume there is a shell program args that contains the following code
echo The first parameter was $1
echo the second was $2
args
uses two shell variables $1 and $2. These are predefined shell variables that will contain the value of the first and second parameters passed to the shell program.
The following are some example runs of args
$ args hello there
The first parameter was hello
the second was there
$ args goodbye
The first parameter was goodbye
the second was
$ args
The first parameter was
the second was
Table 3.2. lists some of the predefined shell variables (for the Bourne shell) that can be used in shell programs to access the program's parameters.
Predefined Shell Variable	Purpose

$0		the name of the shell procedure being executed
$1 thru $9	the first through ninth parameter passed to the shell
		  program (see shift)
$#		the number of parameters passed
$*		used to represent all the parameters passed 
$$		contains the process id number of the current process,
		  often used to create unique temporary files
$?		used to hold the exit status of the last command, a value
		  of 0 means the last command succeeded, 1 means it failed
"$@"		used to represent all the parameters passed, especially if
		  the parameters contain spaces, must use the surrounding ""

	Table 3.2. Predefined Shell Variables (Bourne Shell)
Exercise 3-5. Write a shell program called arguments that tells the user how many command line arguments were passed to it and what they are.
For example:
arguments hello there
arguments accepted 2 command line arguments
they were hello there

Your shell program should use shell variables wherever possible.


Using More than 9 Parameters


The obvious way to display the tenth parameter passed to a shell program would be echo $10. The problem here is that the shell sees $1 and immediately substitutes in the value of the first parameter.

Instead you must use the shell command shift. shift moves all the parameters one parameter to the left i.e. the first parameter disappears, the second parameter becomes the first, the third the second and so on. So after executing the shift command the tenth parameter has become the ninth parameter.

For example:

The following shell program called args2 demonstrates the use of shift
echo parameters are $*
shift
echo parameters are $*
shift
Example use of the script
$ args2 hello there friend
parameters are hello there friend
parameters are there friend

The exit Command


Every UNIX program when it finishes execution returns an exit status. The exit status is used to indicate whether or not the program succeeded.
An exit status of 0 indicates success.
An exit status of anything else indicates failure.
The exit command is used by a shell procedure to change its exit status.
			exit Command Format.

		exit number

	number should be 0 if the shell procedure succeeded.
	  Any other value indicates failure.

	Figure 3.1. The exit Command.
In a Bourne shell program the predefined variable $* is used to hold the exit status of the last command executed.

For example:

The following is code for a shell program called search
!#/bin/sh
# use the -s (silent) option of grep
# first parameter should be a username
# see if the username exists
grep -s $1 /etc/passwd
echo $?

Generally speaking this program will display 0 if the grep found the username in the password file and 1 if it didn't.

The Conditional Commands


A conditional command is a command that makes a choice depending on the outcome of some condition. Shells offer two conditional commands if and case. Both of which are similar to constructs you will have met in other programming languages.

The if Command

			if Command Format.

	if commandt
	then				
		command-list
	fi

	if commandt
	then
		command-list1
	elif command1
		command-list2
	fi

	command-list is a list of 1 or more UNIX commands.
	commandt is any UNIX command.

	Figure 3.2. The if Command.
The main difference with the UNIX if is that it waits on the outcome of a UNIX command.

For example:

	if grep -s $1 /etc/passwd
	then
		echo user $1 was found in password file
	else
		echo user $1 NOT FOUND
	fi
The command grep -s $1 /etc/passwd will search the password file for the word entered as the first parameter. If the command is successful the first echo command will be executed.


The case Command


The other type of conditional command supported by the Bourne shell is the case command. The case command allows the user to compare a single value against multiple other values and when a match is found execute the associated command list.
			case Command Format.

		case value in
			pat1)	command
				command;;
			pat2)	command-list;;
			*)	command-list;;
		esac

		Figure 3.3. case Command Format.
value is compared against each of the patterns signified by pat1.. one by one. If there is a match then the command list associated with that pattern is executed. The ;; symbol is used to signify the end of a command list for a particular pattern.

Once the end of the appropriate command list is reached, execution is restarted at the command directly after the esac. Only the first pattern matched will ever be executed.

The * pattern is optional and is used to match any value and so it acts as the default. (The * should never be the first choice. * matches everything and so the other choices will never get a look in.)

The * is an example of using wildcard characters in the patterns. You can also use the ? and [] wildcard symbols in the patterns. The | symbol can be used as an OR operator for patterns. i.e. pat1 | pat2 will be matched if at least one pattern is a match

For example:

	case $1 in
		hello ) echo hello there;;
		goodbye ) echo why goodbye
			    echo you haven't said hello;;
		T* | *T) echo that word started or ended with a T;;
		[a-z]?) echo that word started with a
			  echo lowercase letter and was
			  echo two letters long;;
		*) echo sorry I don't know that word;;
	esac

	case $# in
		0) echo no parameters;;
		1) echo one parameter;;
		2) echo two parameters;;
		3) echo three parameters;;
	esac

The test Command


Almost every person who has ever written a shell procedure has written one called test and then wondered why it wouldn't work. The reason is that there is already a UNIX program called test. When you enter test at the command line it will typical find the UNIX command before it finds your shell procedure.
The test command is generally used in if statements for testing one or more conditions.

		test Command Format.

	test expr
	[ expr ]

	The [ is actually the name of an executable program (Try the command
	 which [ or whereis [).  When it is executed you are actually running
	 the program called test.

	Figure 3.4. test Command Format.
The symbol expr can be one of the many expressions understood by the test command. test recognises expressions that test for the existence of files, compare the values of integers, compares strings etc. Tables 3.4-3.7 provide a sample of some of the available formats of expr.
	Expression		Result

	-z string		true if the length of string is 0
	-n string		true if the length of string is not zero
	string1 = string2	true if the two strings are identical
	string1 != string2	true if the two strings are NOT identical
	string			true if string is NOT the null string

		Table 3.4. Expressions for testing String Conditions
	Expression		Result

	int1 -eq int2		true if the two integers are equal
	int1 -ne int2		true if the two integers are not equal
	int1 -gt int2		true if int1 is greater than int2
	int1 -ge int2		true if int1 is greater than or equal to int2
	int1 -lt int2		true if int1 is less than int2
	int1 -le int2		true if int1 is less than or equal to int2

		Table 3.5. Expressions for testing Integer Conditions.
	Expression	Result

	-r file		true if file exists and is readable
	-w file		true if file exists and is writable
	-x file		true if file exists and is executable
	-f file		true if file exists and is a regular file,
			 or true if file exists and is not a directory
	-d file		true if file exists and is a directory
	-h file		true if file exists and is a symbolic link
	-c file		true if file exists and is a character special file
	-b file		true if file exists and is a block special file
	-p file		true if file exists and is a named pipe
	-u file		true if file exists and its set user ID bit is set
	-g file		true if file exists and its set group ID bit is set
	-k file		true if file exists and its sticky bit is set
	-s file		true if file exists and has a size greater than 0

		Table 3.6. Expressions for testing File Conditions.
	Expression	Result

	!		the unary negation operator, reverses the result
			 of an expression
	-a		binary AND operator
	-o		binary OR operator
	( expr )	parentheses used for grouping, parentheses have
			 special meaning to the shell so to use them in the
			 test command you must quote them

		Table 3.7. Logical Operators.

Example uses of the test Command


If the first parameter in a shell procedure is hello display a greeting message.
	if test $1 = hello
	then
		echo hello there
	fi
If the first parameter is greater than or equal to the second value (and both are integers) display an appropriate message.
	if [ $1 -gt $2 -o $1 -eq $2 ]
	then
		echo $1 is greater than or equal to $2
	fi
If there are no parameters to this shell procedure display the appropriate message.
	if [ $# -eq 0 ]
	then
		echo You didn\'t enter any parameters
	fi
Exercise 3-6. Rewrite the shell program arguments from exercise 3.5. so that it tests to see if there were any command line arguments. If there weren't any it should display the message No parameters passed. Otherwise it should carry out the actions specified in exercise 3.5.

Exercise 3-7. Write a shell program exists that takes a filename as a command line parameters. If there are no command line parameters it should display a message explaining how to use it. If there are parameters it should test to see if a file with that name exists and display an appropriate message.
For example:
exists /etc/passwd
The file /etc/passwd exists.


Repeated Action Commands


The Bourne shell provides three repeated action commands each of which corresponds to constructs you may have met before in other programming languages

The for command


Execute a list of commands for a specified list of values.
		for Command Format.

	for variable in word1 word2 word3 ... wordn
	do
		command-list
	done

	Figure 3.5. for Command Format.
variable will take on each value in the list word1 word2 word3 ... wordn one value at a time. For each value the commands in command-list will be executed.

For example:

	count=0
	for param in $*	# for every parameter
	do
	  count=`expr $count + 1`	# expr talked about later
	  echo parameter $count is $param
	  # display the number parameter and its value
	done

	echo days of the week are
	for today in sun mon tue wed thu fri sat
	do
	  echo $today
	done
Exercise 3-8. Modify the shell procedure exists written for exercise 3.7 so that it allows the user to enter multiple filenames.
For example:
exists /etc/passwd /home/jonesd temp.dat


The while Command


While the exit status of a command is true continually execute a list of commands.
		while Command Format.

	while command1
	do
		command-list
	done

	Figure 3.6. while Command Format.
command1 is executed and if its exit status is true then the command-list is executed. This is repeated until command1 returns a false exit status.

For example:

	display a countdown from 10 to 0
	count=10
	while [ $count -ge 0 ]
	do
		echo  $count
		count=`expr $count - 1`
	done

The until Command


Repeat a command list until a command returns an exit status of TRUE.
		until Command Format.

	until command1
	do
		command-list
	done

	Figure 3.7. until Command Format.
command1 is executed and while it returns an exit status of FALSE the command-list is executed. This continues until command1 returns an exit status of TRUE.

For example:

	display a countdown from 10 to 0
	count=10
	until [ $count -lt 0 ]
	do
		echo $count
		count=`expr $count - 1`
	done

Some Other Useful Commands


Table 3.8. lists some of the simple UNIX commands you are likely to use when writing shell scripts. The next section talks about some of the more complex ones in detail. With the following use the combination of the examples below and the man pages for each command to work out what it does.
	Command	Purpose

	expr	evaluates logical and arithmetic expressions
	cut	used to extract fields of data from the lines of a file
		 or the output of a command
	sort	sorts its standard input into order
	head	display the first X number of lines of standard input
	tail	display the last X number of lines of standard output
	file	attempts to describe the type of a file
	grep	search for lines containing a character string
	uniq	remove duplicated lines from standard input
	strings	displays any ASCII characters contained in a binary file
	tr	translates specified character(s) into other character(s)
	paste	the opposite to cut, puts lines back together

		Table 3.8. Some other useful commands.

Examples


expr 5 + 6
expr 5 \* 10 + 5
echo "105 - 5 = " `expr 105 \- 5`

cut -d: -f1,6 /etc/passwd
cut -c1-8 /etc/passwd
cut -d: -f1 /etc/passwd	t -d: -f1 /etc/passwd | sort

cat /etc/group		
sort /etc/group	
sort -r /etc/group

cut -d: -f1,6 /etc/passwd | tr : '    '
cut -d: -f1 /etc/passwd | tr [a-z] [A-Z]

Conclusion


As a UNIX Systems Administrator (and throughout the rest of this course) will be called on often to write, debug or change shell scripts. It is important that you understand the concepts and can put them into practice. The more you program in any language the easier it becomes.


Review Questions


3.1. Write a shell procedure called my_sum which accepts a list of numbers and then calculates the sum of all the numbers.
	bash$ my_sum 1 2 3 4 5
	Total = 15
3.2. Modify the my_sum shell script so that it produces output resembling the following.
	csh> my_sum 1 2 3 4 5
	1 + 2 + 3 + 4 + 5 = 15
3.3. Write a shell script called file_type that accepts as parameters the names of files. The script should decide what type of file the file is and inform the user. The shell script should display a help message if no parameters were passed and be able to handle multiple filenames as input.
File types are listed in table 2.1.


Previous | Next

David Jones (author)
Chris Hanson (html 22/08/96)