HRMNotes.htm by Wilf Ratzburg - PAY EQUITY EMPLOYERS' GUIDE (Chapter 2)

Pay Equity Directorate
Anti-Discrimination Programs Branch
Canadian Human Rights Commission

The job evaluation tool... must include the four criteria of

skill,
effort,
responsibility and
working conditions...

and

...reflect the range of work found in a particular organization

...the key elements or "building blocks" of a job evaluation tool...

"criteria",
"components" and
"factors"

...a job evaluation tool... must be capable of measuring all jobs in the organization equitably and consistently

The factors must measure the skill, effort, responsibility and working conditions involved in a job...

...each factor must be gender-neutral

Paired comparison involves comparing each job against every other job, one factor and one job at a time...

...works well if there are only a few jobs in an organization

The point factor method uses factors to define each of the four criteria.

Chapter 2

Phase Two: The Job Comparison Tool

Key points made in this chapter

The job evaluation tool:

as described in the Guide, could be either a point factor or paired comparison (by factor) methodology
must include the four criteria of skill, effort, responsibility and working conditions which are often defined through the use of factors
must reflect the range of work found in a particular organization and capture that organization's values
must make the work visible including those features of work found in jobs performed mainly by women which have been overlooked in the past
may be customized to best reflect a particular organization's set of circumstances
may require creating new factors or adapting existing ones (the methodology for creating factors is covered in this chapter)
must avoid gender bias

Introduction

This chapter discusses the key elements or "building blocks" of a job evaluation tool -- namely "criteria", "components" and "factors" -- and how they relate to each other. It also describes the essential characteristics or requirements of a job evaluation tool, and provides "how to" advice relating to factors that should be useful in developing such a tool.

Essential Characteristics of a Job Evaluation Tool

The following characteristics are essential for a job evaluation tool -- regardless of whether an organization chooses to create, adapt or buy one:

It must be capable of measuring all jobs in the organization equitably and consistently, e.g., from the top to the bottom. It must be able to measure the full range of each job, for example, the interpersonal skill requirements for the Director of Human Resources to those of the cleaner.
It must help make all work requirements visible so they can be listed and evaluated fairly; for example: working conditions that include dust, grease, exposure to disease or dangerous situations.
The factors must measure the skill, effort, responsibility and working conditions involved in a job.
The factors chosen and the items used to measure each factor must be gender-neutral.

The Guidelines have similar provisions (see Section 9 of the Guidelines in appendix B), and say that an employer's job evaluation system should be used in an investigation of any complaint to the Canadian Human Rights Commission if it complies with three conditions, which are, in effect, legislated standards:

it operates without any sexual bias
it is capable of measuring the relative work of all jobs in the establishment; and
it assesses the skill, effort and responsibility and the working conditions determined in accordance with Sections 3 to 8 of the Equal Wages Guidelines.

Most frequently, organizations find their systems do not comply with the first two items above. However, in many cases job evaluation tools can be adapted to better meet these requirements and need not be abandoned.

The Makings of a System provides more details on the design and content of job evaluation tools.

Paired Comparison and Point Factor Comparison

There are a number of job comparison methods. This Guide discusses only two: paired comparison and point factor comparison, with the emphasis on the latter.

Paired comparison involves comparing each job against every other job, one factor and one job at a time. This method works well if there are only a few jobs in an organization. There is no separate standard or scale to compare them to; one job will have more of a particular element of work (e.g., physical effort, mental effort, manual dexterity) than another, and will thus rank higher when the two are compared. Because of its simplicity, paired comparison has traditionally been used by small employers or by large employers that have only a small number of jobs. If the paired comparison approach is used, it should be done factor by factor, with a description of each factor provided to minimize inconsistency in various evaluators' understanding of each factor. See appendix C for more information on creating and implementing this type of methodology.

The point factor method uses factors to define each of the four criteria. For example, a scale for measuring emotional effort could range from working with very difficult people (the high end) to working with courteous people (the low end). Each factor has its own scale, which becomes the standard against which all jobs are compared. Each factor includes one or more variables for measuring an element of work: for example, the degree of disagreeableness and how often an employee would encounter or experience the disagreeable conditions; complexity; or the extent of financial impact. Since each scale is then divided into degrees, it is possible to evaluate jobs in two ways: first against the standard, according to the terms used in the scale; and second against other jobs by comparing each job to every other job rated at the various levels of the scale.

Job Evaluation Criteria, Components and Factors

The diagram below shows how criteria, components and factors relate to each other.

To illustrate these relationships, we have chosen the "Skill" criterion and its components and factors.

Skill Criterion
Component A:	Component B:
Intellectual Skill	Physical Skill
Sample Factors	Sample Factors
Job Knowledge Product Knowledge Contextual Knowledge Communication Interpersonal Skill Analytical Skill Numeric Skill	Sensory Skill Physical Skill

...factors must relate to one of the four legally required criteria: skill, effort, responsibility and working conditions

...the Act defines two components -- "physical skill" and "intellectual skill" for the Skill criterion...

...the Responsibility criterion has four components: responsibility for technical resources, financial resources, human resources, and "other" responsibilities

The Equal Wages Guidelines define specific components for each criterion. For example, the Act defines two components -- "physical skill" and "intellectual skill" for the Skill criterion. Similarly, the Responsibility criterion has four components: responsibility for technical resources, financial resources, human resources, and "other" responsibilities.

As shown in the diagram above, each component has a number of factors associated with it. For example, seven factors appear under "intellectual skill". Each factor is designed to measure or recognize a certain aspect of the intellectual skill component.

In essence, evaluators use the factors to assess the requirements of a job. In any organization, the weight or relative importance of a given factor will vary according to the importance the organization attaches to it. For example, a software engineering company might attach more importance or "value" to jobs that require analytical skills and less to jobs that require interpersonal skills or sensory skills.

The Makings of a System provides thirty sample job evaluation factors. These are not intended to be exhaustive. However, reviewing them may give an organization an opportunity to reflect on what it values in jobs, and to build the appropriate tool for measuring those values. Feedback and participation from people at all levels of the organization will help to create the best tool possible.

The following section provides background information on factors, leading up to the section on choosing appropriate factors (pp. 33 to 45).

A Summary of the Four Criteria

As noted above, factors must relate to one of the four legally required criteria: skill, effort, responsibility and working conditions.

(See the Guidelines, Section 3 to 8 in appendix B.)

The following section provides background information on factors, leading up to the section on choosing appropriate factors (pp. 33 to 45).

The skill criterion measures things according to level of difficulty and things that require training or practice.

Effort measures the mental or physical drain on employees.

Responsibility measures things that have varying degrees of impact or importance to the organization.

Working conditions relate to stress, or bothersome or dangerous work.

A Summary of the Four Criteria

As noted above, factors must relate to one of the four legally required criteria: skill, effort, responsibility and working conditions.

(See the Guidelines, Section 3 to 8 in appendix B.)

1. Skill

The skill criterion measures things according to level of difficulty and things that require training or practice.

Includes two components - mental abilities and physical abilities - required to perform the job
Usually considers such variables as complexity, difficulty and speed.

2. Effort

Effort measures the mental or physical drain on employees.

Refers to the mental or physical demands placed on employees and includes two components: intellectual effort and physical effort
Considers such variables as frequency, duration, exertion, strain, demand, control, and predictability, because these help to quantify the amount of effort required

3. Responsibility

Responsibility measures things that have varying degrees of impact or importance to the organization.

Refers to the importance of certain elements of a job and their potential impact on the organization
Has three components: responsibility for human, technical, and financial resources
Usually includes such variables as importance, size, value, accountability

4. Working Conditions

Working conditions relate to stress, or bothersome or dangerous work.

Refers to the context within which employees are required to perform their jobs
Includes two components: physical conditions and psychological conditions
Usually considers such variables as disagreeableness, negativity, danger, unpredictability of elements, probability, duration and frequency (See box, p. 54.)

The factors must be able to capture the value of all the jobs to be evaluated, in a gender-neutral, objective and consistent manner.

Factors and the Organizational Context

The principle behind the Guide is that organizations will choose factors to fit their own context. Context, in this situation, means values, range of work required, mandate, what the organization is about and what it requires, considering the nature and purpose of the organization and the variety of jobs that must be performed to enable it to carry out its business. The factors must be able to capture the value of all the jobs to be evaluated, in a gender-neutral, objective and consistent manner.

Variables, Ranges and Levels: Promoting Objectivity in Job Evaluations

The two "factor matrices", below, illustrate how variables, ranges and levels contribute to job evaluation. After each matrix, we have included a text-based summary that complements the matrix presentation.

Matrix A: Job Knowledge Factor

DEPTH	BREADTH
	Narrow	Moderate	Broad	Very Broad
Minimal	LEVEL 1	LEVEL 2	LEVEL 3	LEVEL 4
Moderate	LEVEL 2	LEVEL 3	LEVEL 4	LEVEL 5
Extensive	LEVEL 3	LEVEL 4	LEVEL 5	LEVEL 6

Level One

Job requires very little job specific knowledge.
Job requires minimal knowledge of a narrow range of duties and procedures.

Level Two

Job requires a small amount of job-specific knowledge.
Job requires minimal knowledge of a moderately broad range of duties and procedures, or job requires moderate depth of knowledge of a narrow range of job duties and procedures.

Level Three

Job requires a moderate amount of job knowledge.
Job requires moderate knowledge of a moderately broad range of duties and procedures, or job requires extensive knowledge of a narrow range of duties and procedures, or minimal knowledge of a broad range of duties and procedures.

Level four

Job requires a large amount of job knowledge.
Job requires extensive knowledge of a moderately broad range of duties and procedures, or job requires moderate knowledge on a broad range of duties and procedures, or minimal knowledge of an extremely broad range of duties and procedures.

Level Five

Job requires a significant amount of job knowledge.
Job requires extensive knowledge of a broad range of duties and procedures, or moderate knowledge of an extremely broad range of duties and procedures.

Level Six

Job requires a very significant amount of job knowledge.

Job requires extensive knowledge of an extremely broad range of duties and procedures.

Matrix B: Communication Factor

RECEIVING (READING/LISTENING)
PROVIDING (WRITING / SPEAKING)	Simple	Somewhat Complex	Complex	Very Complex
Simple	Level 1	Level 2	Level 3	Level 4
Moderately Complex	Level 2	Level 3	Level 4	Level 5
Very Complex	Level 3	Level 4	Level 5	Level 6

Level One

Job requires little or no communication skill.
Job may require employee to provide simple information and receive simple information.

Level Two

Job requires some communication skill.
Job may require employee to provide moderately complex information and receive simple information, or provide simple information and receive somewhat complex information.

Level Three

Job requires moderate communication skill.
Job may require employee to provide very complex information and receive simple information, or provide moderately complex information and receive somewhat complex information, or provide simple information and receive complex information.

Level Four

Job requires strong communication skills.
Job may require employee to provide very complex information and receive somewhat complex information, or provide moderately complex information and receive complex information, or provide simple information and receive very complex information.

Level Five

Job requires very strong communication skills.
Job may require employee to provide very complex information and receive complex information, or provide complex information and receive very complex information.

Level Six

Job requires extremely strong communication skills.
Job may require employee to provide very complex information and receive very complex information.

A factor typically has two variables associated with it.

Each variable has a range (or scale) associated with it.

1. Variables

A factor typically has two variables associated with it. The variables for the job knowledge factor (Matrix A) represent the depth of knowledge and breadth of knowledge a job requires. Similarly, the variables for the communication factor (Matrix B) depict the requirement to provide or receive information.

2. Ranges

Each variable has a range (or scale) associated with it. A range allows one to measure or compare requirements for different work. For example, Matrix A displays a range from "minimal" to "extensive" for depth and a range of "narrow" to "extremely broad" for breadth. In essence, ranges measure some aspect of a variable. In the case of job knowledge, the ranges measure the extent to which a depth and breadth of knowledge are required. In the case of communication, the ranges measure the degree of complexity of the information that one must provide or receive in doing a particular job.

3. Levels

Levels (or degrees) are clearly displayed on each matrix. A level represents a value that (in the case of Matrix A) is a combination of how much depth and breadth of job knowledge is required to perform a given job. Similarly, for matrix B, a level represents the complexity of the information associated with a job. The essential task in arriving at the appropriate level for a job for a given factor is determining where it would fit in terms of the vertical and horizontal axes. The appropriate level would be where the two points intersect on the factor matrix.

Nil Level

In many factors the bottom level will represent a minimal requirement for a given element. It recognizes that even jobs at this level have a minimum requirement for the elements in question and that must be recognized. In other factors, the bottom level represents jobs that do not have even the slightest amount of the element in question. This suggests that some requirements are present in all jobs to varying degrees, and some elements may be absent from a proportion of the jobs but required in others to varying degrees.

The lowest level in a factor will be scored in one of two ways: points are awarded where this level describes minimal requirements, recognizing actual work demands, or no points are given (a "nil" level) where some jobs in the organization have no demand. It is useful to illustrate this nil requirement, so evaluators can understand where a continuum does begin at zero.

NOTE: Job evaluation is never absolutely precise. A variable must have a range or scale of at least two levels or degrees. An upper limit of six or seven degrees is reasonable. With too many, job evaluation rating committees find it difficult to distinguish clearly enough between jobs.

All of the factors in Making of a System have predesigned scales, each with a number of levels. An organization may have to expand or shrink these scales in order to distinguish between jobs. What is most important is that the scale be able to measure the jobs an organization wants to measure and differentiate the value of the jobs to the degree necessary.

By looking at job factors in terms of their variables, ranges and levels, evaluators can bring a disciplined, uniform, analytical approach to their work. Using either the factor matrix or the text-based approach promotes objectivity in ranking all jobs in relation to each other. Both approaches help the evaluator to focus clearly on the characteristics or requirements relating to the various factors. By assigning a level to a job for each factor, evaluators can establish equivalencies between very different kinds of work.

Points to Remember for Job Evaluation...

...measure jobs relative to one another, and define the measurements

Job evaluation is about measuring work... it is important not to overlook parts of the job. It is just as important to measure only the jobs, not the employee.

A frequent problem for organizations trying to measure jobs is including too many elements of work in one factor.

Points to Remember for Job Evaluation:

a) Measure jobs relative to one another, and define the measurements

In point factor job evaluation, jobs are evaluated against a scale or standard and then compared against each other to determine what each is worth to the organization. To put some boundaries and meaning to the relative terms used in each factor, the organization or the job evaluation rating committee should discuss and, if necessary, define terms such as occasionally, frequently, small, and large. Sometimes this can be accomplished by talking about the types of job demands that are expected to score at the highest level on a factor and those that are expected to score at the lowest level, and why. Another option is to talk about examples from within the organization of responsibilities, skills, physical and mental demands, and working conditions that would be rated at a high level or a low level on a specific factor. It is usually not necessary to define terms such as occasionally and frequently or small versus large with quantitative figures, such as once a week versus everyday, or $10,000 versus $100,000. If the committee decides to be this precise, it should be sure to write the definitions down to ensure they are used consistently in the future and are broad enough to include all kinds of work. Even with less concrete definitions, it is a good idea to write them down for future use.

b) Make work visible

Job evaluation is about measuring work. Therefore when choosing factors it is important not to overlook parts of the job. It is just as important to measure only the jobs, not the employee. Many aspects of work have been left out in the past because they were considered "just part of the job" or intrinsic to employees. Their absence is sometimes defended by the claim that they are not measurable. It may be possible to measure a particular aspect of work, but doing so may require new insights to make it visible and to evaluate it.

For example, the Guidelines give legislative life to the issue of stress. The Guidelines require that stress of the job be measured as a working conditions factor. People may feel that stress is one of those "immeasurable" elements of work. Having a working definition of stress may enable its components to be measured, e.g., stress from unpredictability, from multiple demands, from deadline pressures, or from dealing with people. These are all different ways of examining job stress that are useful in collecting job information and for measuring stress in jobs on a scale.

c) Make scales mutually exclusive, exhaustive and progressive

Constructing scales in a consistent manner will eliminate gaps between levels (i.e., the gap that occurs when a job falls between the end of one level and the beginning of the next). Scales must be exhaustive. For example, if one degree lists supervision of two to four people, and the next, ten to one-hundred people, where should jobs be ranked with supervision of five to nine people?

The levels in the scale cannot overlap, but must be mutually exclusive. For example, one scale cannot indicate at the same time that Level 2 is for jobs with supervision of ten to one-hundred people, and Level 3 for those with supervision of seventy-five to two-hundred.

An error of "omission" occurs where no rules are provided to eliminate the potential confusion that may result when a job has a number of requirements which can be scored at several degrees. How does one choose where to place the job? Another type of violation of this rule occurs when a factor measures elements that are different but might be required by several jobs at varying levels. For example, if a factor such as disagreeable conditions requires frequent travel for Level 2 and frequent exposure to hazards at Level 3, it is difficult to determine how to rank a job that fits in both places. If this is to remain as the definition, very clear rules are necessary about how to evaluate such jobs. The rule may be that the job is automatically given the highest rank: i.e., the most common condition, the most frequent or the most difficult. It is also important to apply such a rule consistently.

A progressive factor is one where higher levels include the lower levels. For example, if Level 1 reflects the requirement to work frequently in very disagreeable conditions, it is likely that the jobs may also require working occasionally in slightly disagreeable conditions. The job gets credit for the higher level even though the lower level is also true. All progressive factors work on the premise that a job which requires a lot of something also requires a little of it but that the reverse is not true. Progressive factors are the only ones in which levels do not need to be mutually exclusive.

d) Factors must have boundaries

A frequent problem for organizations trying to measure jobs is including too many elements of work in one factor. Ideally, for example, the same factor would not measure work hazards, dirt, a small work space, and travel. The organization should know what it is measuring in each case, and have control over the factors. Combining several issues in the same factor will lead to confusion. Furthermore, the cumulative impact of these elements in a job will not be recognized, meaning that some jobs may not get the points they otherwise would if the elements were measured on separate scales. Finally, it is easier to measure issues separately.

In practice, it may be difficult to measure each element of work separately. For instance, the number of factors could be overwhelming. The goal, however, is to be aware of what is being measured and to be as clear as possible. An organization that chooses to measure several issues at once will need to find ways of ensuring consistency and good discipline.

e) Identify equivalencies

Measuring more than one variable or element, as in the majority of factors presented in this manual, creates different paths to the same result, or equivalencies. For example, for a factor that measures physical effort, some jobs may require occasional heavy lifting, while others require constant light lifting. In the first place, the effort is required as a result of the weight without the frequency, and in the second case, it is the frequency and not the weight that causes effort. It would be appropriate to consider the two levels equivalent because a similar amount of effort is required. A job that requires heavy lifting constantly would rank higher than the other two jobs because of the combination of the two elements. Similarly, work that requires counselling others on complex problems would score higher than a job where the relationship with others is limited to obtaining information, and solving simple problems.

When faced with equivalencies, identify them; it is easier to be consistent this way. For example, indicate clearly that "light but continuous physical effort" is equivalent to "occasional heavy physical effort."

Frequently Overlooked Aspects of Women's Work

When determining how to value jobs, consider all the relevant job requirements, including those elements of work that have often been overlooked in the past. There are two questions to ask when deciding whether women's work has been recognized: have the right factors been defined; and has each factor been defined broadly enough?

One problem with factor definition occurs if the factor is too narrow to recognize a complex work demand. For example, for factors that relate to the criteria of effort and working conditions, it is rarely the case that a complex work demand can be effectively valued using one element or variable. It is not just the degree of exertion or drain, but also the duration or frequency, that is important. Where more than one variable is being measured, using a matrix such as the Matrix Presentation on page 25, makes it possible to see the whole work demand at a time.

Equivalencies, which help call attention to elements of work sometimes not recognized in traditional jobs, such as light and continuous physical effort, also become evident. Having this bird's eye view ensures that all appropriate paths to an end result are represented.

Paying attention to frequently overlooked aspects of work will identify those aspects that traditional factors have not recognized. These may require a new factor if existing factors do not relate to the requirement of work. For example, physical effort can be expanded to include light but continuous work in addition to heavy work. Physical effort could also recognize the physical drain caused by lack of movement. Just as sitting for long periods on a plane causes physical fatigue, so too does sitting at one's desk all day. Variety of movement is less fatiguing. But there may not be a factor to address interpersonal skill requirements. In this case, a new factor should be added to measure those types of job requirements.

The following are commonly overlooked aspects of female-dominated jobs (this list is not meant to be exhaustive):

a) Skill

i. Interpersonal skills required for working with children or adults who have problems that require sensitivity and effective communication on numerous levels, including non-verbal communication where voice tone and inflection are important, eye contact, calmness, knowing how to set the right atmosphere

ii. Operating and maintaining various types of equipment, including photocopiers, computers, manufacturing equipment, packaging equipment, diagnosis and monitoring equipment

iii. Hand and finger co-ordination for such things as assembling parts, giving injections, operating equipment such as sewing machines or cash registers, entering information on a computer or diagnosis equipment, or giving therapeutic massages

iv. Records management

v. Formatting documents and designing graphics

b) Effort

i. Lifting either heavy items, or a person who either may be moving or squirming, such as a child having a temper tantrum, or who is fragile, such as an elderly person

ii. Doing a number of tasks simultaneously, which requires a great deal of mental effort to juggle the numerous tasks, and is challenging and fatiguing, e.g., being tuned into two separate conversations, a computer screen, and papers on the desk. (Many women's jobs, such as nurse, clerical worker, service clerk, receptionist, social worker, and teacher, are regularly required to do this type of work.)

iii. Physical effort, which includes fatiguing elements such as restricted movement, awkward working positions, and repeated use of a few muscles (This last element has recently been blamed for causing repetitive strain syndrome in some people such as those doing data input, whose jobs require frequent repetitive movements.)

iv. Concentration for prolonged periods of time

v. The mental challenge of having to accomplish tasks that involve other people's contributions without the corresponding authority or power to ensure those people's compliance

c) Responsibility

i. Protecting confidentiality and handling sensitive information, such as proposed dismissals or lay offs, salary discrepancies, inequities between peers

ii. Having to respond to emergencies in a boss's absence

iii. Managing logistics for meetings, conferences, or organization-wide parties

iv. Caring for people, including providing emotional support, reassurance, and comfort, as well as taking care of bodily needs such as bathing and dietary requirements

v. Training and orienting new staff, including more senior employees

vi. Co-ordinating work flow, including schedules, production processes and materials, supplies, and logistics for meetings, conferences or training

vii. Being accountable for tools or equipment such as computers, photocopiers, diagnosis and monitoring equipment, production equipment, or medical instruments

viii. Contacting or being contacted by people from all levels of other organizations to gather or provide information

d) Working Conditions

i. Emotional demands, such as dealing with death and dying; for example, sensitive and emotionally draining circumstances, such as having to clean a person's body after the person has died but before the body stiffens

ii. Noise from equipment, which may be aggravated by open concept offices or other scenarios where noise is not suppressed or contained

iii. Stress from dealing with hostile, abusive, irrational, upset, ill, or injured people

iv. Exposure to sickness or disease

v. Irregular and unpredictable working hours

vi. Exposure to corrosive substances or other materials that can lead to long-term ailments and chronic disorders, such as breathing problems or skin irritations from cleaning substances

vii. Stress from multiple and often unpredictable demands

Options for Picking Factors

There are five options for achieving the right job evaluation factors for your particular organization while ensuring gender neutrality. The first is choosing from existing material that is already available, including the sample factors provided in Makings of a System. The second option is creating new factors with the help of some suggestions and methods provided in this Guide. The third is taking an existing factor and adapting it to improve its ability to reflect job value ? this may be done with a factor that is currently being used or one from the selection in Makings of a System. The following pages of this Guide give some guidance as to how that might be done. A fourth option is to use a combination of the first three options and the final possibility is to do nothing and maintain an existing set of factors.

Each of the first four options will be explained separately, followed by a discussion of the general theories behind job evaluation factors.

1. Choosing from Existing Factors

An organization can choose factors that measure skill, effort, responsibility and working conditions from existing material such as the 30 sample factors suggested in Makings of a System. When choosing appropriate factors, an organization should consider whether they:

fit in the four main criteria
help to differentiate jobs
do not overlap with other factors
reflect the organization's values
measure job content
avoid gender bias
are clearly written
comply with the Guidelines

Choices made will reflect such considerations. The box on the following page illustrates how two organizations from different industries and with different mandates might need some similar and some dissimilar factors. While this chart does not address the issue of weighting, that is another way in which organizations may customize the way they measure the value of work in their particular context. See chapter 5 for more details on weighting.

2. Creating Factors

This approach requires the most thought and investment of time and may not be necessary for all factors as existing ones may be suitable. However, understanding how factors are created will allow you to determine whether current or potential factors are well written or suitable for use as they are. The methodology used in this Guide is not necessarily used by all organizations, and it is certainly not the only method; we do recommend a disciplined, consistent approach to ensure consistent use of words and a scale that minimizes the potential for bias. Some existing factors may fail on the basis of an internal logic problem, such as gaps in the scale continuum, as opposed to the feature of work that they are meant to measure. If so, understanding how to create a factor may allow the same feature of work to be valued in a slightly different manner.

Possible Factor Combinations
Health Care Organization	Transportation Organization
Skill:	Skill:
*Intellectual Skill*	Intellectual Skill
Job Knowledge Analytical Skill Interpersonal Skill	Job Knowledge Product Knowledge Communication
*Physical Skill*	Physical Skill
Sensory Skill	Physical Skill
Effort:	Effort:
Intellectual Effort	Intellectual Effort
Versatility/Flexibility Creativity with Constraints	Concentration
*Physical Effort*	Physical Effort
Physical Effort	Physical Effort
Responsibility:	Responsibility:
*Technical Responsibility*	Technical Responsibility
Responsibility for Quality Responsibility for Product	Responsibility for Quality
*Financial Responsibility*	Financial Responsibility
Financial Impact Confidentiality	Financial Impact
Human Resources	Human Resources
Serving/Caring for People Responsibility for Others	Responsibility for Others
*Other*	Other
Independence of Action Responsibility for Co-ordinating	Independence of Action Responsibility for Co-ordinating
Work	Work
Working Conditions:	Working Conditions:
Physical Work Environment	Physical Work Environment
Hazards Disagreeable Conditions	Hazards Disagreeable Conditions
Psychological Work Environment	Psychological Work Environment
Stress from Interpersonal Contacts Stress from dealing with the Unpredictable	Stress from Dealing with the Unpredictable Work Scheduling/Travel Effects on Lifestyle

The following steps outline the process used to create the factors found in Makings of a System. They can also be followed to create new factors.

Step 1 Determine the job features to be measured, and which criterion ? either skill, effort, responsibility or working conditions ? they fit.

(For a discussion of these criteria, see A Summary of the Four Criteria, page 23)

As a preliminary step, one must assess whether the factors tentatively selected are likely to reflect the range and features of work in the organization, Take a representative sample of jobs and ask the following questions about the factors that relate to those jobs:

Do they reflect the variety of work, and its features, across the organization?
Does the work (and, therefore, the factors) contain commonly overlooked characteristics of female work? (See p. 30.)

Then try to understand each key feature of the job in terms of the four pay equity criteria. Start by asking job-specific questions based on each of the criteria, to see which criterion best fits the feature of work you wish to measure.

In many instances, fitting the job feature to one of the criteria will be simple. For example, a requirement to write reports falls under skill and may be measured through the communication or job knowledge factors. Perhaps, a job feature can be measured as either a skill or a responsibility. In cases of uncertainty, ask whether, for example, it is the difficulty or the importance that is more relevant, and measure according to that priority.

For example, the job feature that requires interactions with people could be measured in relation to each of the four criteria:

The job feature that requires interacting with people such as clients, customers, patients or suppliers, could be measured, for example, in terms of the ability to read body language, knowledge of when to comfort or discipline someone, when to push for a sale and when to use silence, or how to counsel people through a crisis (the skill criterion).
The stress of interacting with people could be measured through the effect on the employee of dealing all day with complaints, with children who demand attention, or with frustrated and rude customers.
Where interactions help the organization to accomplish its mission (the responsibility criterion), contacts, or responsibility for providing service to people may be appropriate measures. They will value the importance or potential impact of the contact on the organization, the employee's level of accountability, or the nature of the interaction. For example, someone in a supervisory or management role would encourage, motivate, and discipline others.
Interacting with people may require physical effort measured by examining such job duties as lifting a person (remembering, when designing the factor, that a person who doesn't want to be lifted such as a child or patient will be heavier.) Mental effort could be measured in terms of versatility needed; personnel staff, for example, must be able to change roles several times each day, to deal with such things as finances, counselling employees, and negotiating contracts.

While all four of these criteria measure somewhat different features, it would be important to avoid overlap. It would likely be necessary to choose as opposed to using all of them.

Step 2 Ensure the element can be measured

Although you may want to recognize some feature of work, the key to job evaluation is measuring the extent to which the feature is part of, or important to, different jobs. If there is little difference between jobs, you will not be able to use it. An example may help. An organization may believe that its most valuable product is information. It requires that its employees keep that information strictly confidential. Such a responsibility would normally be prominent on the list of what should be evaluated in work. However, all may agree that every job has the same level of responsibility for confidentiality. Since there is no way to differentiate or find different degrees of responsibility for information, it will add nothing to the goal of job equity: developing a hierarchy of jobs.

An element of work (i.e., a job feature) might seem appropriate, but turn out to be difficult to document other than by looking at performance or personal characteristics. However, approaching the question from another perspective could help reveal alternative ways of measuring. For example, measuring the skill involved in problem-solving may be easier than measuring the mental effort involved. Or one could measure the frequency with which problems are likely to occur. Common measures are occasionally or frequently. Another variable might be intensity, or level of challenge, of problems: "frequent problems of minor complexity" could be seen as equivalent to "occasional problems of major complexity".

In other words, the answer to measuring is often the yardstick ? what dimension of that feature of work can be measured? When it comes to lifting, the weight of the items comes to mind as does the frequency with which they are lifted. With contacts, it may be the complexity of content, or the difficulty of obtaining information from the other person.

Some features of work may not suggest measures quite so readily. Problem-solving might be an example. It is a broad concept, and could be a skill or a responsibility. If it is seen as a skill, then it might be measured in terms of complexity, or the variety of problems and the intellectual disciplines required. If it is seen as a responsibility, then it might be measured in terms of significance, probability of impact, value and accountability.

When approaching a feature of work to decide if it is measurable, look first to see if it has a range ? from light to heavy, simple to complex, and so on. See also whether it is more readily measurable through one criterion than another, such as skill versus responsibility.

Step 3 Determine which variables will measure each factor

Variables must be chosen to measure each factor whether they relate to a skill, effort, responsibility or working condition. Variables are such elements as complexity, frequency, importance, or impact. One, two or three variables may be used in conjunction with one another. For example, a variable such as frequency could be used with exertion, or depth of knowledge with breadth of knowledge.

Step 4 Choose or create the number of degrees for each variable

As noted above, variables measure factors. In turn, degrees measure variables. In other words, jobs can be compared because each factor is broken down into variables, and each variable, in turn, is broken down into degrees. The end result is a scale that indicates how much of each variable a given job involves, and thus where that job should be rated according to the factor in question. Degrees progress from least to most: light, medium, heavy; occasionally, frequently, constantly; little, moderate, considerable.

The number of degrees for a given variable depends on how many perceptible differences exist between jobs for that variable. For example, in an organization where most jobs involve lifting all day, the frequency variable would have fewer degrees than in an organization where fewer than half of the jobs involve lifting some of the time. Ideally, each variable will have two to seven degrees. The variables in the matrix below are frequency and exertion, with three and two degrees, respectively.

Step 5 Build a factor matrix. If two or more variables are chosen, a matrix is helpful. (if only one variable is chosen, a matrix is not possible.)

Physical Effort
	Frequency
	Occasionally	Frequently	Constantly
Light	1	2	3
Heavy	2	3	4

The numbers in this chart show levels which are explained in Step-6.

Step 6 Establish equivalencies within the factor

Each time there is a step from one degree to another (e.g., from light to heavy), a higher level is given. For example, Level 1 is given to the slot where both variables are at their lowest. Level 2 is the slot where either (but not both) of the variables has increased one degree over the lowest, and so on.

Some combinations of work requirements will be at the same level, i.e., equivalent, for different reasons. For example, a job could be measured at Level 2 because it involves frequent light physical exertion. Equivalencies reduce the six combinations (boxes or cells) in the matrix above to four factor levels.

Step 7 Check for inclusivity or completeness of the issues and aspects of work covered under the particular factor

When measuring job information, what is missing can cause problems. Gender neutrality requires that the aspects of work in all their complexity be captured through the evaluation process. In the example provided earlier, physical effort was not one dimensional; to avoid gender bias both the frequency and amount (exertion) of physical effort had to be measured.

3. Adapting Factors

In many cases, an existing job evaluation system or particular factors may need only minor adaptations to enable them to recognize the features of work in your organization. Adaptations include lengthening or compressing scales for variables so that the degree of differentiation reflects the differences involved. Adding or changing variables may better capture the desired feature of work. The earlier example of physical effort being not simply a result of exertion but of frequency as well is one type of adaptation.

Adaptation is likely to be an organization's most common approach to factors, as it allows the strengths of existing factors to be maintained and weaknesses to be corrected.

The factors included in Makings of a System can be edited in two ways. The matrix can be edited by adding (or subtracting) one or more levels to the table, which will require the insertion (or deletion) of columns or rows. In addition, it is essential that all equivalencies be properly expressed; the matrix can show whether they have been. The point form text will require additional (or fewer) levels.

4. Combination Option

Some organizations may decide to adapt some of their existing job evaluation factors and create some new ones. Other organizations may decide to purchase a job evaluation system but then adapt some of the factors. Either of these combinations can improve the job evaluation process by ensuring that factors fit the organization, thus improving gender neutrality and compensation patterns.

Some Common Problems with Job Evaluation Factors and Systems

You can use the following list of typical problems to analyze your organization's job evaluation system.

1. The Choice of Elements to Consider

Missing factors: Factors are the basis of valuing; if they do not measure a feature of work, it is not considered, and that can lead to gender bias. For example, a job evaluation system that does not have a factor to address co-ordination skills such as finger, hand and foot movements will not be able to rate accurately the jobs in which they are required.

Missing aspects of a factor: Factors that do not measure all relevant aspects of a variable can result in gender bias. A physical effort factor that measures only heavy exertion and ignores repetitive light exertion or restricted movements is an example.

2. Language

Jargon: Words or phrases that are obscure or highly specialized can easily be misunderstood. For example, Level 4 of a factor "requires mathematical, financial, logical concepts to analyze more complex..."
Unclear or confusing language: If the words in various parts of the job evaluation system, such as the information-gathering tool or the factor degrees, do not have one clear meaning, employees and evaluators will be unable to answer questions and evaluate jobs. If words have several possible meanings and they are interpreted differently by each person, problems, including gender bias, may result. The words must direct participants to a reasonable and consistent answer. Use clear and precise language.
Language is too complex for intended users: Overly complex language or long, passive sentences can cause difficulties for some employees. If people do not understand what they are being asked, the information and the evaluation will not be fair. This point relates to more than the issue of gender bias. Therefore, it is an important consideration.
Using examples that create an image of jobs for one sex only: Words used in the factors may create images of jobs in people's minds. For example, the job evaluation rating committee may read physical effort in a job questionnaire and think of heavy lifting jobs. This will be especially true if the examples given apply only to heavy lifting jobs. Organizations will find it helpful to give examples of equipment, tasks, and so on that relate to the physical effort in women's as well as men's jobs.

3. The Logic or Flow of Factors

Overlapping factors: Overlap occurs when the jobs that score high (or low) on one particular factor will also always score high (or low) on another factor. This may be either partial overlap, where parts of one factor overlap with another, or total overlap, where two factors measure the same thing. Some overlap in measuring work is inevitable, but it can result in bias when the elements of work that are counted twice favour jobs of one sex or another. For example, if an organization used the factors interpersonal skills and contact with people and they both tended to recognize the same elements, these would be counted twice. And if female jobs alone were benefiting from the count, gender bias would result. If an organization wants to give substantial weight to a particular aspect of work, it should do so through factor weighting rather than by measuring the same thing twice.
Factor scales that are not mutually exclusive or progressive: A job may fit the scale in more than one place for different reasons. Within a factor, degrees increase along a continuum, and higher levels include lower levels. For example, reading complex information assumes that jobs also require the reading of simple information. This is not a problem so long as the committee consistently credits the higher (most frequent or most important) level.

A problem arises when the levels do not measure the same skill on a continuum. For example, if Level 3 refers to reading complex information, and Level 4 refers to communicating with senior management, or Level 2 refers to working in somewhat dirty conditions, while Level 3 refers to travelling frequently, the levels are not measuring the same element and a job could fit the scale at different levels for different reasons.

Using the factor matrix to create factors will help maintain a continuum. (See The Matrix Presentation on p. 25.) Employers should ensure that there is only one spot on a scale to rate each degree of a variable.

Factor scales that are not exhaustive: For the physical effort factor, a slot for all levels of exertion is needed to reflect all job requirement possibilities. If the scale does not have a level to describe a particular degree of exertion, there is no way of valuing the job requirement.

Too many variables: It is important to limit the number of variables being measured by a factor. If a factor measures too many variables, confusion over what is being measured may result.

Wrong variables are used: For example, it is inappropriate to use the frequency variable to measure a skill factor (frequency is a measurement for effort). Instead, depth of knowledge (a variable that measures for skill should be used). See Makings of a System.

Factors that do not measure what they purport to measure: If a factor is described as a skill factor but measures responsibility, or is supposed to be about problem solving but is really about supervisory responsibility, gender bias could result if the confusion caused by any ambiguity benefits one gender over the other. Regardless, it is a good idea to ensure a logical relationship between the factor chosen and the criterion (i.e., skill, effort, responsibility or working conditions) being measured and the particular terms being used in the variables (e.g., frequency, complexity, and so on).

4. Organizational Context

Factors that do not fit the organizational context: Gender bias must always be analyzed within the context of the individual organization. An organization cannot assume that the same factors which fit another organization will also meet its needs. A telephone company will want to measure different elements of work than an organization operating a grain elevator. Many of the aspects of work needed to accomplish the phone company's goals will be left out if it tries to apply factors developed for a grain elevator, and vice versa, considering the range of work in each organization.

Scales with too few or too many levels: If the scale does not have the appropriate number of levels, proper and effective differentiation of jobs will be difficult and likely not effective. Job evaluation is not scientific. If the scales are too finely differentiated, differences will be difficult to perceive and consistent ranking will not be possible. If the scales are too broadly or too generally differentiated, some jobs are unfairly lumped with others below where they should be ranked. This can lead to gender bias, particularly if there is a gender pattern. For example, if female-oriented aspects of work have broadly defined factor scales of only three levels, and male-oriented factor scales have finely differentiated scales with six or seven levels, male jobs may have a better chance of getting a higher rating, and the results may not be fair.

Nil level: Zero points may be gained from the lowest degree in a factor; only higher degrees accord any recognition to demands in the work. If a factor has nil level then it is possible to gain points only from the other levels. This means the real number of levels is reduced by one. If the factors that have a nil level are more likely to favour or disadvantage jobs in one gender, this could present a gender bias.

5. Link with Job Information

Missing job information: Items or tasks are not made visible or job duties are missing from the job description or questionnaire. Complete and effective job information is essential for accurate job evaluation results. Gender bias can result from incomplete job information. What is not "visible" or "captured" will not be evaluated or compensated.

6. Does the Job Evaluation Tool Minimize Gender Bias?

Some Practical Questions to Ask:

Do the factors measure skill, effort, responsibility or working conditions (is each properly defined for the particular criterion it is meant to recognize)?
Are factors defined broadly enough to capture the range of work in this organization?
Are there any missing factors that would be required to recognized features of work that are necessary to this particular organization?
Do the factors cover the typically overlooked factors in female-dominated work?
Is the wording of the factors understandable?
Is the language too complex?
If examples are given, is there a gender bias?
Are the levels in each factor mutually exclusive, progressive and exhaustive?
Is there any overlap between factors?
Do the levels follow a logical flow?
Are the scales differentiated enough to recognize differences in work?
Is there a gender trend to factors having a nil level?
Is there a trend in the length of factor scales (do some have long or finely differentiated scales and others short or broadly differentiated scales), and does this trend show a relationship to gender?

Avoiding Gender Bias in the Job Evaluation Tool

Gender neutrality is crucial whether an organization creates its own factors, adapts some of those from this Guide or elsewhere, or purchases a job evaluation system.

Three suggestions to help meet the challenge of minimizing gender bias in the job evaluation tool:

1. Ensure the job comparison tool uses the four criteria ? skill, effort, responsibility, and working conditions ? to determine the value of the jobs.

2. Ensure the factors chosen are free of gender bias. (Refer to the checklist on previous page.)

3. Make sure the factor levels are free of gender bias language and that they capture all the possible work requirements.

Bias is often caused by leaving out certain aspects of jobs. It can be hard to see what is missing; one way to minimize bias is to carefully determine what work is necessary to accomplish the organization's goals and mission, and to make sure the factors measure those elements of work.

Example:

An organization must determine how to define skill, effort, responsibility and work conditions in a way that reflects the organization's values and the range of work performed and is free of gender bias. The decision-makers (who may include management, union personnel, an employee committee, and a consultant) should review the information produced at the first step (see p. 36) and then do some brainstorming. What are the skill requirements and how can they be defined as intellectual and physical skill requirements? Do jobs require coordination, analytical thinking, communication skills, and the like? What about the effort requirements as defined by intellectual and physical effort? This needs to be done for all four criteria and must capture all the components listed in the Guidelines.

Next, define and expand the factors chosen to describe what they are designed to measure. For example, what is meant by communication, or analytical skill? The factors must always be designed to measure job requirements, not performance or individual abilities.

After the brainstorming exercise, think about what may be missing (the material provided in The Makings of a System, the sample factors and questionnaire items, may be a start to exploring this), and review the material provided in Chapter 3, the information-gathering stage, below.

Factors must value all aspects of work consistently. Aspects of work commonly associated with men, such as physical effort, must also be measured in female jobs even though they may not look the same: lifting people instead of objects, or lifting light objects continuously, rather than heavy objects occasionally. Also, aspects of women's work that have gone unnoticed, such as providing efficient and courteous service to the public, knowing emergency procedures, performing numerous tasks simultaneously, or informal training and co-ordinating the work of others, must be included.