Previous | Next

Section 16 `CRON`, ACCOUNTING, AND QUOTAS

Objectives

This section aims to introduce you to

the cron daemon,
the location and format of the crontab files,
the crontab command,
mechanisms for limiting which users can use cron.
the type of information stored by the UNIX operating systems accounting systems,
the commands last, lastcomm and other commands that access the accounting information,
the syslog system that provides a central distribution point for system status and information messages,
the BSD disk quota system, and
other methods for limiting the amount of resources users may consume.

Introduction

The facilities introduced in this section allow the Systems Administrator to

schedule actions to be carried out automatically at set and regular times, The cron system.
track and record the actions carried out by the system's users, and Accounting systems and syslog.
limit the amount of resources that are consumed by these users. Quotas and other schemes.

A number of the responsibilities of a System Administrator are automated tasks that must be carried out at the regular times every day, week or hour. Examples include, early every morning freeing up disk space by deleting entries in the /tmp directory or running the various security scripts described in the previous section.

Most of these responsibilities require no human interaction other than to start the command. Rather than have the Administrator start these jobs manually, UNIX provides a mechanism that will automatically carry out certain tasks at set times.

crond is a system daemon that will execute other programs at the times specified in the appropriate configuration files. The daemon is normally started in one of the system's start up files (/etc/rc*).

The configuration files that control the operation of cron are called crontab files (cron tables). These files contain information about the time, date and command to execute. Different versions of UNIX store these crontab files in different locations and use a slightly different format.

On most systems the crontab files can only be modified using the crontab command. Even though the crontab files are text files they should not be modified using an editor.

Table 16.1 summarises the various components of the cron system.

	Component	 Purpose

	crond		 the daemon that reads the configuration files
			   and runs the commands
	crontab		 the command used to create the configuration files
	cron.log	 a file used to store the output and results of
			   commands executed by crond
/usr/spool/cron/crontabs the directory in which the configuration files are kept
 cron.allow  cron.deny	 files used to control which users can create their
			   own crontab entries (only available on some systems)

		Table 16.1. cron Files, Commands and Daemons.

`crontab` Files

The crontab files are stored under the system's spool directory (usually either /usr/spool or /var/spool) in the directory cron/crontab. SysV and some BSD systems will have individual crontab files for every user, while some BSD systems combine crontab entries for all users into one crontab file.

The crontab files are text files with one entry per line. Each entry contains either six (SysV) or seven (some BSD systems) fields separated by white space. Table 16.2 summarises the fields of a cron configuration file.

The additional field on some BSD sysetms indicates the username of the user the entry belongs to. This field isn't needed under SysV systems as each user has their own crontab file.

	Field		Purpose

	minute		minute of hour, 00 to 59
	hour		hour of the day, 00 to 23 (military time)
	day		day of the month, 1 to 31
	month		month of the year, 1 to 12
	weekday		day of the week, either 1 to 7, or 0 to 6,
	username	name of user executing command
	command		command to execute

		Table 16.2. Fields of a cron Entry.

Weekdays usually start with Sunday as either 0 or 1. However a few systems start the week on Monday. The manual pages on cron or crontab should list the format used a particular system.

Comments can be used and are indicated using the # symbol. Anything that appears after a # symbol until the end of that line is considered a comment and is ignored by crond.

The first five fields use any one of the following formats.

an asterix that matches all possible values,
a single integer that matches that exact value,
a list of integers separated by commas (no spaces) used to match any one of the values
two integers separated by a dash (a range) used to match any value within the range.

Some systems allow lists to contain ranges (e.g. 1,2,5-10,12), however most do not.

For example:

	every hour (when minutes=0) display Cuckoo Cuckoo on the system console
	0 * * * * echo Cuckoo Cuckoo > /dev/console

	at half past the hour, between 9 and 5, for every day of January which
	is a Sunday, Wednesday or Saturday, append the date to the file date.file
	30 9-17  * 1 0,3,6 echo `date` >> /date.file

The commands are run by the crond daemon. This means that they are not associated with any particular terminal and so there is no place for them to get standard input or put standard output and standard error. This means if the output of these commands are to be used it must be redirected. Specific log files (like date.file from the example above) or electronic mail are common destinations.

Some systems provide a file cron.log that is used to store the output of commands executed by crond.

Exercise 16-1. Using the manual pages discover the following

a) What format does your system's crontab files use?
b) Does your system have separate crontab files for every user?
c) What day starts the week on your system?

Exercise 16-2. Using the format of your system write crontab entries for the following.

a) remove all the contents of the directory /tmp at 5:00am every morning
b) execute a shell script /root/weekly.job every Wednesday
c) run the program /root/summary at 3, 6 and 9 pm for the first five days of a month

Allow and Deny

Some systems provide the ability for all users to make use of the cron system. On some systems the Systems Administrator may wish to restrict access to cron. Access can be restricted on most systems using using the files cron.allow and cron.deny.

These files contain a list of usernames, one username to a line. These files specify who can and who can't use the cron system using the following rules if a username appears in cron.allow that user can use cron, if a username appears in cron.deny that user cannot use cron, if neither file exists only root can use cron, and if cron.allow does not exist and cron.deny does but is empty then everyone can use cron.

The location of these files can differ from system to system.

The `crontab` Command

crontab files should not be modified using an editor. Instead the crontab command is provided. The actual format of the command may change between systems but the following provides an introduction.

1. crontab [file] or
2. crontab [-e | -r | -l ] [username]

Version 1 is used to replace an existing crontab file with the contents of standard input or the specified file.

Version 2 makes use of one of the following command line options

-e allows the user to edit the crontab file using an editor (the command will perform some additional actions to make it safe to do so)
-r remove the user's crontab file
-l display the user's crontab file onto standard output

By default all actions are carried out on the user's own crontab file. Only the root user can specify another username and modify that user's crontab file.

For example:

	create a normal file that contains my crontab file
	crontab -l > my.crontab
	edit the normal file using an editor and make the necessary modifications
	vi my.crontab
	replace the existing crontab file with the new one
	crontab my.crontab

Exercise 16-3. What would the contents of the cron.allow/cron.deny files be so that

a) only root may make use of cron
b) only root and one other user may make use of cron

Exercise 16-4. Add entries to root's crontab to run the shell script passwdck (created as an answer to one of the exercises from Section 13) every second day at 7am. Output of the script should be mailed to the root user.

Quotas and Process Resource Limits

Quotas and process resource limits allow the Systems Administrator to place limits on the amount of resources a user can consume. The main use of these systems is to ensure that individual users do not consume the majority of a resource and as a result prevent other users from making use of the system. For example, it is unlikely for it to be okay for an individual user to consume 95% of disk space. Limits can also be put in place to prevent run away process from bringing your system down.

The choice of whether or not to implement quotas and process resource limits is up to the Systems Administrator. The decision may depend on

the maturity of your users, and
the importance of your system being available.

Quotas is another area in which BSD and SysV based Unices differ. The standard SysV quota system is very limited and does not offer as much functionality as that offered by the BSD based quota system. Many recent releases of SysV UNIX offer a BSD like disk quota system.

Process Resource Limits

Both the C shell and the Korn shell support commands that implement an ineffective form of process resource limits. The type of process resources these systems can limit are specified in Table 16.3. The C shell uses the limit command and the Korn shell the ulimit command to view the limits.

Both systems use the concept of

soft limits, and
Default limits applied when a process is created that can be increased by the user up until the hard limit is reached.
hard limits.
System wide hard limits that cannot be bypassed. The hard limits are usually specified in the kernel and sometimes cannot be modified by the Systems Administrator.

	C Shell		Korn Shell	Resource

	cputime	time	total		accumulated CPU time
	filesize	file		largest possible file size
	datasize	data		size of a process' data segment
	stacksize	stack		size of a process' stack
	coredumpsize	coredump	size of a core file
	memoryuse	memory		amount of memory

		Table 16.3. Possible Process Resource Limits.

Process limits suffer from a number of problems

the hard limits are hard wired into the kernel,
users may change their own soft limits, and
sometimes the limits aren't actually enforced.

The BSD Disk Quota System

The BSD disk quota system allows the Systems Administrator to limit

the number of disk blocks a user can consume, and
the number of i-nodes a user owns (every file needs one i-node).

Under the BSD system disk quotas are handled on a per user, per file system basis. This means disk quotas can be set individually for each user on each file system.

For example:
The user jonesd might have one set of quotas for the /home file system and another set of quotas for the /usr/local file system. The user munstert can have a different quota under /usr/local and no quota under /home.

The system allows the specification of two limits (similar to process resource limits)

a hard limit, and
This is the absolute limit. The user will not be allowed to exceed this limit. The file system will simply refuse to carry out any request that increases the size.
a soft limit.
The soft limit serves as a warning. If the user passes the soft limit they will receive a warning message. After a set number of warnings the soft limit will begin to act like a hard limit.

How it Works

There are various commands associated with the BSD disk quota system that are summarised in Table 16.4.

	Command		Purpose

	quota		display a user's disk usage and quota
	quotacheck	ensure that quota records match disk usage
	quotaon		turn quotas on
	quotaoff	turn quotas off
	repquota	report on implemented quotas
	edquota		modify a user's quota

		Table 16.4. Summary of Quota Commands.

For quotas to work the following must be carried out

the file system and the kernel must support quotas,
the quotaon command must be used to turn quotas on (usually during one of the systems start up scripts),
the file systems that are to have quotas working must have their entries in the /etc/fstab or /etc/vfstab files modified to include quotas in the options, and
entries must be added to the quota control files to set the quotas for individual users.

The quota information for each user is stored in a quota control file usually called quotas. Some systems use quota.user and quota.group to set quotas on individual users and groups.

Each partition for which quotas have been enabled will have its own quota control file. The file is kept in the root directory of the partition and contains the following information

the current disk usage for each user, and
the current hard and soft limits for each user.

quotas is a binary file and is not in a human readable form. Individual users can view their quotas by using the quota command. The quotas for all users can be viewed by using the repquota command. Typically only the root user can view another user's quotas.

For example:

	A system has the following /etc/fstab file
	/dev/hdb1        swap           swap        defaults
	/dev/hda2        /              ext2        defaults
	/dev/hdb2        /home          ext2        defaults,usrquota
	/dev/hda1        /home/ftp/pub  ext2        defaults
	none             /proc          proc        defaults
	Quotas have been turned on for the /home parition (notice) the additional
	  option usrquota.  This means there should be a quotas (or on this
	  system quota.user) file in the root directory of that partition.
	bash$ ls  -l /home/quota.user
	-rw------- 1 root students 164368 Sep 21 13:37 /home/quota.user
	bash$ quota jonesd
	Disk quotas for user jonesd (uid 975):
	File system  blocks quota limit   grace   files   quota   limit   grace
	/dev/hdb2    7       9    16               6*      5      10    none
	This user has a very small quota and is actually over the soft limit on
	  the number of files allowed (notice the 6*)

Quotas are set for individual users by root using the edquota command. This command enters the default editor (defined by the shell variable EDITOR) and allows the root user to modify the the assigned quotas. When the editor is exited the new modified quotas will be added to the quotas file.

For example:

	The following provides an example of what the user sees when they
	  use edquota
	bash$ edquota jonesd
	Quotas for user jonesd:
	/dev/hdb2: blocks in use: 7, limits (soft = 9, hard = 15)
	        inodes in use: 6, limits (soft = 5, hard = 10)
	The root user can now modify these numbers to increase or decrease
	  the user's quota.

The quotacheck command is used to ensure that the record of current disk usage stored in the quotas file is correct. When quotacheck is executed it searches throughout the file system and summarises how many disk blocks and i-nodes each individual user owns and updates the statistics stored in the quotas file.

Users adding or deleting files at the same time quotacheck is being executed can result in inconsistent results. For this reason the quotacheck command should only be executed when there are no users on the system. As a result it is normally executed during system startup before users can login.

Installing Quotas

The process of installing quotas is listed below.

Reconfigure and recompile the kernel
The file system code within the kernel must support quotas. If it doesn't the kernel will have to be reconfigured and recompiled so that it does. This will be possible only on some systems, others simply don't support the BSD quota system.
Modify the /etc/rc script
Quotas can be turned on or off at any time. It is usual to turn quotas on at boot time using one of the /etc/rc* files. On some systems the quotacheck command will be execute before hand (this command can take a long time to execute)
For example:
```
		echo -n 'checking quotas:' > /dev/console
		/etc/quotacheck -a -p > /dev/console 2>&1
		/etc/quotaon -a
```
Modify the /etc/fstab or /etc/vfstab file.
Each partition for which quotas are to be installed must be mounted with the appropriate option. Normally this should be done when the partition is first mounted. This is controlled by the fstab or vfstab files.
For example:
```
	The following example is of an /etc/vfstab file from
	a SysVR4 machine.
			/dev/sd0a	/	4.2	rw 1 1
		to
			/dev/sd0a	/	4.2	rq 1 1
	The change from w to q is a change from write to write with quotas.
	
```
Create the quota control file.
Create an empty file called quotas on the root directory of all the file systems which are to have quotas applied.
```
	touch quotas	do this as root
	chmod u=rw quotas
	
```
Set each user's quota.
Use the edquota command to set the quotas for each user.

Exercise 16-5. If your system supports them turn quotas on for the file system that contains your users' home directories.

Exercise 16-6. Create a new user and set some very strict quotas that allow a very low number of files and blocks. Experiment with the user account to see what happens when the soft and hard limits are exceeded.

BSD Accounting

Most UNIX systems automatically keep log files that summarise what is being done with various parts of the system. The majority of the information is appended onto a variety of accounting files. Most systems supply a number of commands that turn accounting on or off, display the accounting information in a (sometimes barely) human readable format, and summarise and shorten the accounting files.

Both the BSD and SysV accounting systems store similar information but use different commands to summarise and display the information. SysV accounting relies on a greater number of shell scripts and C programs. SysV accounting will not be examined in this section.

The system administrator has two responsibilities with regards to accounting

deciding how to use the information, and
Keeping the logging information doesn't make sense unless you use it. Uses may include documentation to management stating the need for more equipment, charging users for use of the system and as a method to track what the users are actually doing.
pruning the files.
The accounting/log files grow continually and should be pruned occasionally by the administrator otherwise they can grow to consume a lot of disk space.

Accounting on some systems automatically ceases if the file system becomes more than 95% full.

The BSD accounting information is recorded by the by the kernel appending information to a specific accounting file. A number of specific commands are provided to examine the contents of those files. Table 16.5 summarises what information is kept, in what files and what commands can be used to examine the information.

	Information	Commands	Files

	processes	accton, sa,	specified by accton command line,
			 lastcomm	  usually /usr/adm/acct *
	current users	w, who		/etc/utmp
	connect time	ac, last	/usr/adm/wtmp
	disk usage	quot, du	no file
	printer usage	pac		defined by af entry in printers
					  printcap entry
			*the file needs to exists beforehand

		Table 16.5. Summary of BSD Accounting Information.

The files that are written to by the accounting system will continue to grow and take up disk space. They should regularly be pruned. The administrator must also decide whether or not to archive these files. The simplest method to deal with these accounting files is to have entries in the crontab so that the system will automatically take care of the problem.

For example:

	Summarise the accounting file, this produces a much smaller file.
	# summarise /usr/adm/acct file into /usr/adm/savacct
	0 1 * * 1-6 /etc/sa -s
	
	Truncate the wtmp file to nothing
	0 2 1 * *   cp /dev/null /usr/adm/wtmp

The administrator must decide whether or not to save the old version of the accounting files for future use before truncating them.

Process Accounting

Process accounting is started by the accton command. It stores the following information about every process executed by the operating system.

executable file name,
CPU time used in user and system time,
how long the process took,
when the process started,
the process' user and group IDs,
memory usage,
the number of characters read and written,
number of disk I/O blocks read and written,
the terminal that started the process, and
various flags about the process and its exit status.

Process accounting is usually started during the system startup scripts with a line something like

	if [ -f /var/adm/acct ]
	then
		accton /var/adm/acct
		echo -n '  accounting' > /dev/console
	fi

The information is appended onto the file specified in the accton command and is usually /var/adm/acct (on some systems /usr/adm/acct). This file becomes very lengthy reasonably quickly. The sa command can be used to produce a file /var/adm/savacct that is formed by merging the records in the acct file.

This information can be examined using the

lastcomm command, and
lastcomm can be used to examine that data by individual command or user.
the sa command.
On systems that support it the sa command is used to produce usage reports and summarise the accounting files. Reports generated by the sa command produce a wide range of information depending on the command line switches used.

Connect Time

The times when users log in and out are stored in the binary files

wtmp, and
This file is used to record the log in and log out time of every user session on the computer. The information is accessed by the last and ac commands.
utmp.
The utmp file is used to record the users that are currently logged onto the system. The information is accessed by a variety of commands including w and who.

`syslog`

Both BSD and SVR4 (the latest release of SysV) support the syslog system. This system is used by a variety of programs and sub-systems to direct status information and error messages to one central location, the syslogd daemon. Prior to the existence of syslogd these messages would be sent to a variety of files and locations all over the file hierarchy.

syslogd provides a central distribution point for status information and error messages and provides the Systems Administrator with the ability to direct these messages to be

appended to specific files,
forwarded to a different machine, or
sent to the consoles of all or some of the logged in users.

Where the messages are sent is controlled by the syslog configuration file /etc/syslog.conf. syslog.conf is a text file with the following format

	facility.level[;facility.level...]  destination

facility describes where the information is coming from. Some of the possible sources are listed in Table 16.6. level describes the severity of the information (Table 16.7) and destination specifies where the message is to be sent.

		Value	Source

		kern	the kernel
		mail	the mail system
		lpr	the printing system
		daemon	the various system daemons
		auth	the login authentication system

			Table 16.6. Facilities that may use syslog.


		Level	Description

		emerg	a system panic
		alert	an error that requires immediate action
		crit	a critical error
		err	errors
		warn	warnings
		notice	non-critical messages
		info	informative messages
		debug	debugging information

			Table 16.7. Levels of syslog messages.

For example:

	The following is an example /etc/syslog.conf taken from a Linux box.
	#
	# NOTE: YOU HAVE TO USE TABS HERE - NOT SPACES.
	#
	
	kern.*                           /home/megan/kern.messages
	*.=info;*.=notice                 /usr/adm/messages
	*.=debug                          /usr/adm/debug
	*.warn                            /usr/adm/syslog

Exercise 16.7. Examine the syslog.conf file on your system (if it supports it) and the various log files the syslogd sends the messages.

Exercise 16.8. Modify the syslog.conf file so that the most serious messages are sent to the system console.

Conclusions

The crond daemon is used in conjunction with crontab files to schedule the automatic execution of commands at specific, regular times.

The UNIX operating system provides a number of systems that allow the Systems Adminstrator to

observe what is and has been done with the system, and
limit the percentage of resources any particular user may use.

This section has introduced you to

the cron system,
the BSD accounting system,
the BSD disk quota system,
the various ineffective process limit schemes, and
the syslog system.

Review Questions

16.1. List and discuss the components of the cron system.

16.2. Describe the effects of the following crontab entry

		* * 1 1,6,12 0-3 echo this day is 'date'>> date.log

16.3. Describe how the BSD disk quota system works.

16.4. What is the purpose of the syslog system?

Previous | Next

David Jones (author)
Chris Hanson (html 10/09/96)

Section 16

CRON, ACCOUNTING, AND QUOTAS

crontab Files

Allow and Deny

The crontab Command

How it Works

Installing Quotas

Process Accounting

Connect Time

`CRON`, ACCOUNTING, AND QUOTAS

`crontab` Files

The `crontab` Command