All ideas in this book are based on a mixture of Yin’s research(Yin is a Ph.D. in experimental psychology) over the past twelve years, a case study method course taught at MIT(Massachusetts Institute Technology) for five years and discussion with many scholars interested in case study research. Those ideas are represented in all chapters of the book that represent the all phases of method of research: problem definition, design, data collection, data analysis, composition and reporting.

1. The Content of the Book

Yin defines the case study as an empirical inquiry that investigate a contemporary phenomenon within its real-life context when the boundaries between phenomenon within its real-life context and when in which multiple sources of evidence are used. More over he described the variations in case studies as a research strategy. Case studies includes a case study and multiple-case studies; can include, and even be limited to quantitative evidence, and case studies have a distinctive a place in evaluation, at least have four different applications in evaluation research. Those applications are: to explain the causal links in real-life intervention that are to complex for the survey or experimental strategies; to describe the real-life context in which an intervention occurred; to explore those situations in which the intervention being evaluated has no clear, single set outcomes; to give benefit to evaluation in descriptive mode.

Yin pointed out that there was a misconception in the past if case studies were appropriate for the exploratory phase of investigation only. Comparing case studies to other research strategies(experiment, survey, history, case study, archival analysis, etc) Yin suggest to use the most appropriate view of different strategies that is a pluralistic view . It means that all strategy can used for all three purposes: exploratory, descriptive, or explanation. For each strategy Yin differentiate based on the (a) form of research question; (b) requires control over behavioral events; and (c) the degree of focus on contemporary events. In general, case studies use ‘how’ and ’why’ of research questions without requires control and focuses on contemporary events,

Research Design Single and multiple case studies

After defined research questions, the next task is to design the case study plan or research design. It is the logic that links the data to be collected and the conclusions to be drawn to the initial questions of a study. A research design in case studies should have five components: (1) a study’s questions, (2) its propositions, (3) its units of analysis, (4) the logic linking the data have been collected, and (5) the criteria for interpreting the findings. The new state-of-art knowledge of the last two are provided in this book and described below.

Generalizing from case study to the theory.

Two categories of generalizing result from case study to theory pointed out by Yin(1989, p.38) are statistical analytic generalization and statistical generalization. In statistical generalization, an inference is made about a population on the basis of empirical data collected about a sample. But this method less relevant to the case studies. On the other hand, the method of generalization is analytic generalization in which a previously developed theory is used as template with which to compare the empirical results of case study. If two or more cases one shown to support the same theory, replication maybe claimed. Graphically, this type of generalization is shown as a level one of inference(see. figure 1). A good case study investigator should make the effort to develop this theoretical framework, no matter whether the study is to be explanatory, descriptive, or exploratory. The use of theory, in doing case studies, is not only an immense aid in defining the appropriate research design and data collection, but also becomes the main vehicle for generalizing the results of the case study.

The case study investigators must maximize four aspects of the quality of any design:

- construct validity: establishing correct operational measures for the concept being

studied;

- internal validity(for explanatory or causal case studies only), and not for descriptive or

exploratory studies): establishing a causal relationship, whereby certain conditions are

shown to lead to other conditions, as distinguished from spurious relationship

- external validity establishing the domain to which a study's findings can be

generalized; and

- reliability: demonstrating that the operations of a study- such as the data collection

procedures- can be repeated, with the same results

Those four tests may be considered relevant in judging the quality of a research design. In designing and doing case studies, various tactics are available to deal with these tests. Some of the tactics occur during the data collection, data analysis, or compositional phases of the research.

There are four steps should be prepared before collect the data in case studies. Those steps are: prior skill of investigators, plan intensive training, develop a case study protocol, and conduct a pilot study case.

Six common sources of evidence can be the focus of data collection for case studies: documentation, archival records, interviews, direct observations, participant-observation, and physical artifacts. In addition, in order to have high-quality case study uses three principles of data collection: using multiple source of evidence, creating a case study database, and maintaining a chain of evidence. This ways will help to deal with the problems of construct validity and reliability.

This chapter has presented several important strategies for analyzing case studies.

1. The potential analytic difficulties can be reduced if an investigator has a general strategy for analyzing the data in a preliminary sense, as a prelude to developing a systematic sense of what is worth analyzing and how it should be analyzed.
2. Three strategies(pattern-matching, explanation-building, and time-series analysis) are effective ways of laying the groundwork for high-quality case studies. For all them, a similar replication logic should be applied if the study involves multiple cases(thereby gaining external validity), but important comparisons to rival propositions and threats to internal validity should be made within each individual case. Three other strategies(analyzing embedded units, making repeated observations, and doing case surveys) are incomplete ways of doing case study analysis.

The reporting of a case study can take written or oral forms. Steps need to be followed in the compositional process:

• identifying the audience for report,
• developing the compositional structure, and
• following certain procedures.

• Each chapter has exercises in which the questions refer to main topics in that chapter and some summaries.
• The number of examples are 40 illustrative case studies.
• The book is well-structured due to some chapters refer to steps on research(introduction, designing, processing, analyzing, and reporting).
• There is a note describes two new importance topics are added in revision edition, first the critical role of theory in designing case studies and in generalizing form them, and later how to determine the number of case studies to be used in a multiple case study.
• There are indexes(author and subject)
• In terms of English, this book easy to read.

• The limitations of the book are that it is not intended as teaching device.
• There is no a glossary
• The size of letter small.

Case study as one of the approaches that often used in developmental research, especially for research question HOW and WHY. This book really useful for developmental researcher if they use case studies approach in terms of the ways:

• how to generalize findings become a theory
• how to make a good research question
• how to make a good research design
• how to collect and analyzed data in case studies
• how to make a report

1. This book is a good guidance either for investigators or students who are trying to do case studies in a thorough and rigorous way.
2. For the novices, to get the idea of Case Study Research, you should practice (Yin,

1989):

• do a very simple case study research
• do it with a very simple research question or simple methodology
• get an idea about the potential difficulties

1. This book is a good reference for reader who want to find the critical role of theory from designing case studies into generalizing from them(see chapter 2., 38-40) and how to determine the number of case studies to be used in a multiple-case study(see chapter 2,52-59)

Yin, R.K. (1989). Case study research: Design and methods(Revised Edition). London: Sage [H62.Y56 ,ISBN 0-8039-3471-8(pbk)]

Yin, R.K. (1994). Case study research: Design and methods(2^nd Edition). Beverly Hills, CA: Sage [H62.Yin HLRBR ,ISBN 0-8039-5662-2

There is an introduction chapter that has explanations about why the book was written, who the book is for, what is news in the book, how the book is organized, how to use the book, and how the book was formatively evaluated. The main content of the book is described in other six chapters that are summarized bellow.

Tessmer(1993) define formative evaluation as a judgement of the strengths and weakness of instruction in its development stages, for purposes of revising the instruction to improve its effectiveness and appeal. Moreover, Tessmer points out that formative evaluation is the ‘quality control’ step that helps guarantee that the finished product will succeed in its intended environments. Formative evaluation consists of identifying problems and weakness in instruction, making revision based on evaluation data, and trying out the revised version to confirm the success of revisions and identify any further weakness.

Tessmer categorized formative evaluation into four steps: expert review, one-to-one evaluation, small group and field test. Ideally, the evaluator would conduct expert and one-to-one evaluation first, revise the instruction, and then conduct a small group evaluation and ‘polish’ the instruction once again. The final step is then to field test the instruction in the learning environments for which it was intended(see figure. 1).

The general planning process of formative evaluation are: determine the goals of evaluation, determine the evaluators, obtain a complete task analysis of the instruction, describe the learning environment, determine the media characteristics of the instruction, outline the information sought from the evaluation, choose parts of the instruction for evaluation, select the stages and subjects for the evaluation, select data gathering methods and tools for each stage, and make the evaluation plan.

Figure 2 . is a layer of formative evaluation that contains the number and degree of evaluation stages used in the project. Ideally, one do as much formative evaluation as his/her can. Realistically, one often have to select and design the layers of formative evaluation that he can use, and do the best evaluation he can within project limitation, to aim ‘excellence within boundaries’. Moreover , from the figure 2. can be seen , that the closer a design project is to completion, the more resistance there is to revision, so that the earlier the formative evaluation is completed the better the chance that its revision recommendations will be accepted and implemented.

Expert review is a intrinsic evaluation of the instruction, meaning that the instruction or the product are evaluated in terms of intrinsic merits such as content accuracy, design or technical quality. Expert review involves an expert reviewing the rough version of the product or instruction to determine its strengths and weakness. In this review, a specialist is given a version of the instruction to evaluate and is asked a set of predetermined questions about it.

There are two advantages of expert review , first the experts are people who have knowledge and experiences, and later some times be can inexpensive if suitable experts are readily available. On the other hand, the disadvantages of expert review are it can not provide performance or opinions from a learner’s point of view and it can be costly if suitable experts are not readily available.

In order to plan expert review you should answer two important questions:

• What information do I want to learn from the reviews?. You should specify the information goals. The information can be about content, teaching, implementation, technical, design expertise, and testing expertise. One way to ask yourself the following questions: What’s new about the instruction?, what could make this instruction fail?, about what aspects do you feel most unsure?, and what changes can you make?
• Who do you choose for an expert review?. In formative evaluation generally, the experts are specialist, people who have special knowledge, skill, or experience with regard to the content, features or audience of the instruction. They could be:
• Subject Matter Experts(SME) that review the content currency, accuracy, completeness, realism, and range;
• Teaching experts who evaluate learnability and effectiveness;
• Subject sophisticates who evaluate content and learning effectiveness;
• Instructional Design Expert who review the Instructional Design concept (adequacy task analysis, clarity, and completeness of learning objectives)
• Other experts such as editors.

After specify the information goals and choose the experts, next steps of expert review process involves: choosing the review formats; preparing the review questions; designing data collection tools; preparing the experts; managing the actual review; and closing the review effectively.

The goal of analyze data is to organize the information to make the revision decisions easier. If more than one experts involved, the following procedures can be used: (1)review one expert’s comments, rejecting ones that would lead to pointless;(2) summarize and list the expert’s comments; (3) review anothers expert’s comments and notes areas of agreement and disagreements between the first and second experts; and (4) repeat the process for all experts involved.

To make revision after expert review, designers use the result of data analysis. But if they are still not sure what revision should be made to the specific objective, it may help to use the second opinion principle: ask another experts about it.

There are two problems may arise, first the reluctant expert(some experts can be rather uncooperative during an expert review)and later the ‘subject matter expert’ mountain (some experts are unable to communicate their expertise during an evaluation). Yet, those problem will be covered with well-prepared questions, instruments, and process management of evaluation.

What alternative to expert review?

Self evaluation and panel review are alternative to expert review. Self evaluation means designer as an evaluator or an expert. It has several benefits but with special detail criteria of evaluation. Panel review means some designers have brought several experts together to discuss the proposed instruction. The advantage of it is save time and the disadvantage is sometimes it can generate unfocused and unproductive session.

One-to-one evaluation may be the most recognized and used type of formative evaluation. It is also referred to as developmental testing. One-to-one evaluations are excellent for identifying errors and problems in rough drafts of instructional materials. It different from expert review because learners are used as subjects, so that the focus of the evaluation is upon clarity, ease of use, interest, and effectiveness of the instructions to learners. Revision can be made after each one-to-one is completed, but some revisions will be delayed until a ‘second opinion’ is obtained from more one-to-one subjects or group evaluations.

Variations of the classic method of conducting the evaluation: two-on-one evaluation and think-aloud protocols. Two-on-one evaluation means two learners as evaluator and designer as moderator the discussion between them and takes notes. With think –aloud protocols evaluation method, learners have to describe their thought process after they use the instruction or product.

Small group evaluations usually follow one-to-one and expert evaluations, and are used to confirm changes made in these and to generate further reversionary data. The number of people usually 6-20 people. Designer as an observer and the learners work through the instruction with minimal interaction with the designer. Small group evaluation intended for evaluate instructional errors and weakness. Small group environments are similar to real-world situations in certain importance elements. Alternative forms of small

group formative evaluation: Evaluation meetings and computer journals.

Field test is a ‘situated evaluation’ , where instruction is evaluated in the same environments in which it will be used when it is finished. It has purposes such as: confirm the revisions made in previous formative evaluations, generate final revision suggestions, and investigate the effectiveness of the instruction. Field tests are used in actual situations that contain all of the elements of future learning environments.

Alternative forms of formative evaluation or extended testing are extended field testing, continuos assessment and rapid prototyping. Extended field testing is a type of continuos field test in which evaluations and revisions are conducted on a regular basis while the product being used. Continuos assessment is used to periodically survey trainees’ workplace competence. Finally, rapid prototyping is both an alternative form of evaluation and instructional design. A working portion of the final product is developed and immediately implemented in the field, where user input is used to revise the final product and design the rest of it. This method involves learners and experts early in the design and evaluation process and moves into field testing the instruction early in the design phase. This method may be particularly suitable for computer based instruction.

• The state of the art or the advanced method offered by this book is relatively new. Published in 1993, it is still the most recently book in the arena formative evaluation.
• This book emphasis on:

• Develop right questions in evaluation
• Formative evaluation for all types of media such as text, video, multi media, hypertext, workshop and lectures.
• Interpersonal issues and problems due to formative evaluation can be intensely interpersonal encounter between subject and evaluator
• The way to consider and make revision is part of every chapter – and it is a critical and difficult issue that has received little coverage elsewhere
• Outlines some alternatives forms of formative evaluation
• This book has an introduction chapter, ‘a note to my readers about this book’; a glossary; an index ; an appendix that has checklist that you may use to self-evaluate your own formative evaluation plan before your conduct; a well-structured contents and an easy-read language.
• Quality and number of "how to do" suggestions

• In the end of each chapter, except chapter one, there is an example how to use each type of evaluation. Type of media that is chosen to be evaluated is a workshop.

According to me, there are not many weakness in this book. A reader who interested in the history of formative evaluation will miss this aspect. And those who want to get the statistics and methods for evaluation needs to use another books.

This book is useful for the arena of education and training. It can be used either as a reference work or a textbook for formative evaluation course.

Formative evaluation is very useful for type one of developmental research that is developing a product. As its function is to evaluate instructional product, so formative evaluation used as a methodology in the development stage. One of the alternative form of formative evaluation, the rapid prototyping, is useful for type one of developmental research in case the product is computer-based.

For anyone who may be involved in developing instructional materials such as texts, lectures, and multimedia this books is very good to used as a reference for doing formative evaluation. This includes educators, trainers, students, or professional both novices and experienced. For researcher who are doing research with type of question WHAT … is really recommended to use this book.

The summation below might be make more clear how to do formative evaluation or process of formative evaluation whatever types(Expert review, One-to-one evaluation, small-group or field test) are used :

1. Preparation includes:

• description of the context;
• formulation of main evaluation questions;
• decision on evaluation approaches;,
• choosing activities, number of respondents, situations; and
• constructing evaluation plan.

b. Data gathering includes:

• clarifying intentions with lesson materials;
• adapting or developing instruments;
• selecting and inviting respondents; and
• data collection.

c. Data processing includes:

• interpreting data; and
• making overview of revision suggestions

d. Reporting includes making overview of revision decisions.

(a developmental research page )

Tessmer, Martin. 1993. Planning and Conducting FORMATIVE EVALUATION.

Improving the quality of Education and Training. London: Kogan Page.

[ISBN: 0 7494 0801 4.]