BSU 510 Assignment
A Usability Assessment Test
for the The Fireworks MX interactive teaching tool
http://www.oocities.org/tfischer5/fireworks_home.htm#return

Conducted
by
Thomas Fischer
April 2006
ITEDU 510
Ball State University
Muncie, Indiana .
Introduction

Purpose Participants Location
Procedures Results Recommendations References
Pre-Test Questionnaire
Post-Test Questionnaire
Introduction

Instructor based tutorials, particularly in a technology based course, have the potential to be effective and flexible teaching and learning tools which are necessary for technology based, multi-leveled classrooms. Because most software programs are designed with many layers of complicated built-in tutorials, students often ignore these tools as a resource.

The Fireworks MX masking tutorial is an example of an interactive teaching tool designed by Thomas Fischer, a technology education teacher at Hopedale JR./SR. High School in Hopedale, MA. This web-based tutorial was designed using a Windows XP environment, as well as "Fireworks MX……" Dreamweaver MX was used as the web authoring software and Fireworks MX as the The intended purpose of this web based tool is to simplify the learning process as well as allow the student to learn in a self-paced, hands-on setting in which the teacher may not be present. Masking was chosen as the learning goal for two reasons. The technique of masking is an often misunderstood yet critical application as it relates to effective graphic and web design. In addition, masking involves subsets within the tools menu that are relevant in other design techniques.

Back to Top
Purpose

The usability test that was performed for this teaching tool has provided valuable information in regards to improvements that could be made to make the tool more effective. Suggestions made during the post-testing questioning section provided insights that will make the teaching tool clearer in its language.

Back to Top
Participants

Six participants were chosen from three separate graphic design courses. The courses were Communications 1, Communications 2 and Independent Study Communications. Two students from each course level were chosen. This sampling method was used in order to assure various skill levels were represented in the sampling group.

Participant Background
1 Communications 1
2 Communications 1
3 Communications 2
4 Communications 2
5 Independent Study Communications
6 Independent Study Communications
Back to Top

Location

The testing took place in a 16 station computer lab. The lab was chosen because this particular lab is used as a communications classroom and is representative of the environment in which the student tutorial would be used. Students not involved in the testing were present in the room during the testing as part of a normal communications class. Again, the reason for this was to simulate, as accurately as possible, the environment in which the student tutorial would be used.

Back to Top

Procedures

Participants were given a verbal explanation by the evaluator as to the purpose of the test. They were then told to follow the instruction on the tutorial to the best of their abilities and that all necessary materials were on a designated desk in the center of the room. Each test was applied separately and the evaluator observed each participant as they navigated through the learning tool and completed the assignment. A pre-test questionnaire was filled out by each participant prior to the test. Each participant was told that there was no time limit, however, each participant was timed by the evaluator. At the completion of the test, each participant was asked to fill out a post-test questionnaire.

Back to Top
Pre-Test Questionnaire
1.Circle one:
I am currently enrolled in:
Communications 1
Communications 2
Independent Study Communications
2. Circle one:
I am:
not familiar with
somewhat familiar with
very familiar with
the concept of digital masking.
3. Circle one:
I am:
not familiar with
somewhat familiar with
very familiar with
Macromedia Fireworks MX
4. Circle one:
I am:
not familiar with
somewhat familiar with
very familiar with
Web based, self-paced tutorials
Back to Top
Usability Study Post-test questionnaire

1.Circle one:
I understood all of the tasks that I was required to accomplish
I understood some the tasks that I was required to accomplish
I did not understand most of the tasks I was required to accomplish

2. Circle one:
I was able to navigate the tutorial and encountered minimal problems
I was able to navigate the tutorial and encountered a few problems
I was able to navigate the tutorial with but encountered many problems
I was unable to navigate the tutorial

3. I thought the hands on masking exercise was:
Circle one:
Very useful
Somewhat useful
Not useful

4. I thought the directions in the first masking section were:
circle one:
Very clear
Needed some clarification
Not clear at all

5. I thought the directions in the second masking section were:
circle one:
Very clear
Needed some clarification
Not clear at all

6. Circle one:
I was / was not able to accomplish the masking assignment in section one.

7. Circle one:
I was / was not able to accomplish the masking assignment in section two.

8. What suggestions would you make that would make the masking tutorial easier to use?

9. Would you prefer to team with a classmate when doing an on-line tutorial?

Four of the six participants understood all the tasks that they were expected to accomplish and two understood some of the tasks. The most confusing task was the mask layering technique.

Three of the participants were able to navigate the tutorial with minimal or no problems while three encountered some problems. The problems were centered around some confusing screenshots.

Five of the six participants found the hands on cutting exercise useful.

Four of the six participants found the directions in the first masking section to be clear and two said they needed some clarification.

Three of the six participants found the directions in the second masking section to be clear, one said it needed some clarification and two participants said the directions in the second masking section were not clear at all.


Five of the six participants were able to accomplish the masking assignment in section one. One participant was not able to accomplish the masking assignment in section one.


Four of the six participants were able to accomplish the masking assignment in section two.Two were not able to accomplish this task.

 


Results

Pre-Test Questionnaire

I am currently enrolled in: Communications 1 (2) Communications 2 (2) Independent Study Communications (2)
I am: not familiar with
the concept of digital masking.
(3)
somewhat familiar with the concept of digital masking.
(2)
very familiar with the concept of digital masking.
(1)
I am: not familiar with
Macromedia Fireworks MX
(1)
somewhat familiar with Macromedia Fireworks MX
(3)
very familiar with
Macromedia Fireworks MX
(2)
I am: not familiar with
Web based, self-paced tutorials
(0)
somewhat familiar with Web based, self-paced tutorials
(3)
very familiar with
Web based, self-paced tutorials
(3)
Back to Top

Recommendations

The results made from observations as well as post-test questionnaire responses suggest that all participants found the hands on cutting activity a very effective means of describing the concept of masking. Also, the screen shots provided in the tutorial were seen as effective but could have been a bit clearer in a few instances. For example, the screen shot which asks the user to choose the mask tool does not point specifically to the mask icon. Suggestions for fixing this ranged from circling the icon to moving the text over the icon.

In addition, a technical glitch was found in the lesson 3 anchor link. The second line of the anchor does not bring the user to the lesson 3 portion of the tutorial. An additional recommendation included adding the lesson number to the heading of each section.

Five of the six participants would have preferred to work with a partner during the tutorial.

 

 

References

Schulze, Patti Macromedia Fireworks MX Training From the Source.Macromedia Press, 2003

http://www.macromedia.com/