Objectives| Introduction| Shell Functions| Input/Output for Shell Programs| Trapping Signals| Regular Expressions| ex, vi and Regular Expressions| awk| Perl| Conclusion| Review Questions
Before getting started lets be certain about the terminology that is to be used in relation to shell programs. Table 4.1 lists the terminology that will be used throughout this course in relation to shell programs. Other people may use slightly different terms.
Term Explanation Shell program an executable file that contains UNIX and shell commands and can be interpreted by a UNIX shell Shell script same as a shell program Shell procedure same as a shell program shell function similar to a function in C or Pascal, only used within a shell program (introduced below) Table 4.1. Terminology for shell scripts and functions.
Bourne Shell Function Syntax name() { command-list } Figure 4.1. Bourne Shell Function Syntax.
Some of the differences include:
#!/bin/sh # an example shell function called hello hello() { # display the function's 1st param echo hello there $1 # display all function parameters echo $* echo $0 local_var="hello" } # call the function and pass to it all the # programs parameters hello $* # show that there is no scope rules echo "Local variable value = " $local_var # call the fucntion but pass it no parameters hello
For example:
temp() { return 1 } if temp then echo succeeded else echo failed fi
An example use of the function would be
file_type /etc/passwd
echo "/etc/passwd is a $type."
read Command Format. read variable-list variable-list is a list of one or more shell variables. The first word read is placed into the first variable, the second into the second and so on. Any excess words are placed into the last shell variable. read always returns an exit status of zero unless an end of file condition is met. With interactive input from the keyboard this is when thekey combination is used. Figure 4.2. read Command Format.
read first second echo "Variable first contains " $first echo "Variable second contains " $secondSome example runs of input
input run the shell script hello there my friend input some data Variable first contains hello Variable second contains there my friend input hello Variable first contains hello Variable second containsAssume there exists a file call input_file with the following contents
input < input_file Variable first contains hello Variable second contains thereAssume the following is in a shell procedure
while read whole_line do echo I just got the line $whole_line done < /etc/passwdThis shell procedure will go through the /etc/passwd file a line at a time. The read command continues to succeed until end of file is reached.
Using \c to supress the terminating newline character will only work on some versions of UNIX. The alternative to using \c on most systems is a -n switch for the echo command.
For example:
echo "Enter a number ==> \c" read numberUse the -n switch to surpress the newline character
echo -n "Enter a number ==> " read number
Character Purpose \b display the backspace character \c display the line without the terminating newline \f do a form feed \n display the newline character \r display the carriage return character \t display the tab character \\ display the backslash character \nnn display any character with the ASCII value signified by nnn (a 1 to 3 digit octal number starting with 0) Table 4.2. echo Escape Characters.
The UNIX operating system generates a number of different signals (Table 4.3. lists some of them). Each signal has an associated unique identifying number and a symbolic name. Table 4.2 lists the signals defined by a UNIX machine using a SVR4 version of UNIX.
Each process has an a default behaviour for every signal. When it receives a signal it carries out this default behaviour. The most common default actions are
kill Command Format. kill [ -signal ] pid ... Sends the signal specified by the number signal to the process identified with process identifier pid. The kill command will handle a list of process identifiers. By default kill sends signal number 15 (the TERM signal). Figure 4.3. kill Command Format.The ps command lists processes, their process identifiers and a variety of other information.
For example:
ps PID TTY STAT TIME COMMAND 26796 pp0 S 0:01 -bash the user's login shell 30434 pp0 R 0:00 ps the process for ps kill 26796this will have the affect of logging the user out as the TERM signal has been sent to the login shell which will terminate on receiving it.
For example:
Another example is the case where you don't want the user to be able to quit out of a shell procedure. In this case you want signals like SIGTERM to be ignored.
Symbolic Name Number Purpose SIGHUP 1 hangup SIGINT 2 interrupt (rubout) SIGQUIT 3 quit (ASCII FS) SIGILL 4 illegal instruction (not reset when caught) SIGTRAP 5 trace trap (not reset when caught) SIGIOT 6 IOT instruction SIGABRT 6 used by abort, replace SIGIOT in the future SIGEMT 7 EMT instruction SIGFPE 8 floating point exception SIGKILL 9 kill (cannot be caught or ignored) SIGBUS 10 bus error SIGSEGV 11 segmentation violation SIGSYS 12 bad argument to system call SIGPIPE 13 write on a pipe with no one to read it SIGALRM 14 alarm clock SIGTERM 15 software termination signal from kill SIGUSR1 16 user defined signal 1 SIGUSR2 17 user defined signal 2 SIGCLD 18 child status change SIGCHLD 18 child status change alias (POSIX) SIGPWR 19 power-fail restart SIGWINCH 20 window size change SIGURG 21 urgent socket condition SIGPOLL 22 pollable event occured SIGIO 22 socket I/O possible (SIGPOLL alias) SIGSTOP 23 stop (cannot be caught or ignored) SIGTSTP 24 user stop requested from tty SIGCONT 25 stopped process has been continued SIGTTIN 26 background tty read attempted SIGTTOU 27 background tty write attempted SIGVTALRM 28 virtual timer expired SIGPROF 29 profiling timer expired SIGXCPU 30 exceeded cpu limit SIGXFSZ 31 exceeded file size limit Table 4.3. Signals on a SYSVR4 Box. trap Command Format. trap [commands] [signals] trap with no parameters displays a list of current trap assignments. signals a list of signals to change the default action for commands is one or more commands that will be executed when ever one of the listed signals is received If commands is the null string then the specified signals are ignored. If only signals is used the list of signals will be reset back to the original default actions. Figure 4.4. trap Command Format.For example:
ps execute ps to find out the pid of login shell PID TTY STAT TIME COMMAND 26796 pp0 S 0:01 -bash 30434 pp0 R 0:00 ps trap "echo I received signal 2; echo goodbye" 2everytime the shell receives signal 2 it should perform the echo commands
signals& run the shell script in the background kill 34256 send signal 15 to the signals process Signals received signal 15
Regular expressions are recognised, in one form or another, by the following UNIX commands ed, ex, sed, awk, grep, egrep, expr and even vi. Regular expressions are used by these commands to match a sequence of characters.
The power in regular expressions comes when they are combined. The primitives from Table 4.4 can be combined together to perform complex matches. Table 4.5 provides a number of examples.
Symbols Purpose c any character other than \ [ . * ^ ] $ (or some subset of these depending on implementation) matches itself \ remove any special meaning from a character . match any one character ^ match the start of the line when it is the first character in the RE $ matches the end of the line when it is the last character in the RE * match zero or more matches of previous character or expression [chars] match any one character from the string chars [^chars]match any one character NOT in the string chars Table 4.4. Summary of Regular Expressions.
Example RE What it matches unix matches the work unix and nothing else [Uu]nix matches any word starting with a u or a U followed by nix [^aeiouAEIOU]* matches any sequence of characters that does not contain a vowel ^abc$ matches lines that contain abc only hel. match any word starting with hel followed by one letter e.g. help hell hel< Table 4.5. Example Regular Expressions.Exercise 4-5. Describe the text that the following REs will match.
A regular expression surrounded by \( \) is considered tagged (the \ character is included because the () characters have special meaning to the shell) and is stored into the next register. The first tagged RE will be placed into register 1, the second into register 2 and so on. The value stored in each register is accessed by \n where n is the number of the register.
For example:
Symbol Meaning + matches one or more occurrences of the previous RE ? matches zero or one occurrences of the previous RE | matches either of two REs separated by a | or both REs separated by a new line () used to group an RE so that * + ? | can be applied to it Table 4.6. Extensions for Full REs.
Symbol Meaning \{n\} match exactly n occurrences of a single character RE \{n,\} match at least n occurrences of a single character RE \{n,m\} match between n and m occurrences of a single character RE Table 4.7. System V Extensions to Limited REs.For example:
Our main interest in ex here is its ability to use regular expressions to perform complex manipulation of text. Particularly interesting is the ability to use these ex commands from vi. Similar commands can also be used under ed and sed.
You will probably already have used a number of ex commands during your normal use of vi. Whenever you enter : and then some command (e.g. :w :wq :q) you are entering an ex command.
For example:
Address Purpose . addresses the current line $ addresses the last line of the file number addresses the line which is number lines from the start of the file letter addresses the line which has been tagged with the label corresponding with the lower case alphabetical character letter* /expression/ addresses the first line which matches the regular expression expression by searching FORWARD from the current line ?expression? addresses the first line which matches the regular expression expression by searching BACKWARD from the current line address+number addresses the line which is number lines FORWARD from address address-number addresses the line which is number lines BACKWARD from address address1,address2 specifies the range from address1 to address2 , stands for the pair 1,$ i.e. the whole file, from the first line to the last line ; stands for the pair .,$ i.e. from the current line to the end of the file * lines are tagged using the k command outlined below Table 4.8. Specifying Addresses for Regular Expressions.
ex Command Format. address command count address as specified in table 4.6 command as specified in table 4.7 count how many times to perform the command count defaults to 1 Figure 4.5. ex Command Format.The buffer referred to in Table 4.9 (for the d command) could be one of the 27 buffers maintained by ex and vi for the purpose of storing text.
There is an unnamed buffer that is used to store text from a normal delete operation. There are then 26 named buffers (referred to by the names a through to z).
For example:
Command Purpose line a append text after line range co line copy lines specified by range to just after line range d buffer count delete lines specified by range and count. Place them into buffer if specfied range j count join text from specifed lines into one line n edit the next file specified in the command line q quit line r file read the contents of file and insert it after line sh start up another shell range s/pat1/rep1/options count replace all patterns matching the RE pat1 in the specified range with rep1 (will only work on the first occurence of pat1 for each line unless g is used as an option u undo the previous change range w file write the specified lines to the file. w>>file appends the lines to the file Table 4.9. Sample of ex commands.Exercise 4-7. Do the following using both vi and sed
The following reading provides you with an introduction to this powerful tool. You will notice that the syntax of awk has much in common with that of the C programming language.
For example:
awk 'BEGIN { FS=":"; print "Usernames matching '$1' are.." } /^'$1'/ { print $1; X = X + 1 } END { print "There were", X, " of them." }' $2
For example:
jones:david:5:55 blow:joe:10:100Write an awk script to produce a file called final.dat that displays the information taken from results.dat in the following format:
jones, david 60 blow, joe 110Assume: a1Mark is out of 10 and is worth 10% of the final result. examMark is out of 90 and is worth 90% of the mark.
user 0 1 2 Group 0 has no users root daemon uucp uucpa Group 1 has 4 users Group 2 has no users
Here's the beginning of the description from the Perl man page:
Perl is an interpreted language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information. It's also a good language for many system management tasks. The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant, minimal). It combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. (Language historians will also note some vestiges of csh, Pascal, and even BASIC-PLUS.) Expression syntax corresponds quite closely to C expression syntax. Unlike most UNIX utilities, Perl does not arbitrarily limit the size of your data--if you've got the memory, Perl can slurp in your whole file as a single string. Recursion is of unlimited depth. And the hash tables used by associative arrays grow as necessary to prevent degraded performance.
Perl uses sophisticated pattern matching techniques to scan large amounts of data very quickly. Although optimized for scanning text, Perl can also deal with binary data, and can make dbm files look like associative arrays (where dbm is available). Setuid Perl scripts are safer than C programs through a dataflow tracing mechanism which prevents many stupid security holes. If you have a problem that would ordinarily use sed or awk or sh, but it exceeds their capabilities or must run a little faster, and you don't want to write the silly thing in C, then Perl may be for you. There are also translators to turn your sed and awk scripts into Perl scripts.
4.1. Write a shell program called mycp. It is to be a replacement for the cp command but instead of simply copying the file it should first check to see if the destination file exists. If it does it should ask the user whether or not they
4.2. The existing cp program supports the following type of command.
cp file1 file2 file3 directory
That is copying more than one file into a destination directory. Modify your mycp command from the previous question to allow this type of syntax.
4.3. It is often the case that specific users on a system continually use too much disk space. There are a number of solutions to this problem including quotas (talked about in a later section).
Implement another solution to this problem that has the following characteristics.
jonesd 50000 okellys 10
David Jones (author)