Running Head: EDID 6508 DEVELOPING INSTRUCTIONAL MEDIA

EDID 6508- Developing Instructional Media

Name: Genevieve Cox

Assignment Part 3: Section: Evaluation of Multimedia Project

Course Facilitator: LeRoy Hill

University of the West Indies Open Campus

Date April 25, 2017

EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
2
Executive Summary

The report which follows was done to fulfill the requirement of inviting two colleagues

from the course EDID 6508 Developing Instructional Media. This evaluation to experience the

usability of the multimedia project created by Collaborative Group 3 on the topic of the

Respiratory System using Captivate 8. The evaluation criteria included rating on a scale of 1 to 4

with I denoting incomplete, and 4 denoting complete coverage of all the assignment elements of

the rating scale set by the instructor. A Voice Thread was prepared and is presented to provide an

overview of the program. It is located at: https://voicethread.com/share/9016126/

The major findings from the evaluation indicated were difficulties in navigating beyond

the first multiple choice question, and issues presented with navigation and content and

organization, and the absence of a Help button.

The group members include :

Gloria Butcher - Test engineer

Genevieve Cox - Group lead and script developer

Jane Pierre - Copy editor

Rupatee Rajkumar Singh - Script writer and editor

Sean Thomas Multimedia Engineer

Rachel Wagstaffe Graphic Designer

EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
3
Table of Contents

Contents
Purpose of the Study....................................................................................................... 3
Description of the product and its context of use :..................................................................4
The usability goals....................................................................................................... 5
Methods:...................................................................................................................... 5
Summary of Findings...................................................................................................... 5
Discussion on findings..................................................................................................... 6
Conclusion.................................................................................................................... 6
Appendix: Multimedia Evaluation Reports from Nyeisha and Stephanie........................................7
EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
4
Purpose of the Study

The review is comprised of the following:

Examination of the story board or planning sheet for the design of the project.
Organization of the content existing in the project is logical and menu paths are clear.
Originality of ideas presented in the project to offer new insights to the learner.
Copyright acknowledgement and documentation of artefacts used in the project.
Format Platform Transferability, in terms of how well the project works on Windows

or Macintosh, or mobile devices.

Subject knowledge in terms of the appropriateness and clearness of the information

presented to the audience.

Graphical design of visual portrayal of illustrations reinforces the text for high impact

delivery.
Mechanics in terms of how well the project elements work together.
Screen design in terms of the balance and layout of the design elements viewed on the

screen to enable the learner to find and navigate to new information.

User Enhancements in terms of the presence of audio, video or suitable graphical

icons.

Evaluative Information Details:

Date of tests: Tests were conducted during the period April 19th to 25th 2017.

Number of participants: 2 participants (Nyeisha George Minott and Stephanie

Lionel).

Length of tests: Less than 1 hour.

Location: Online using students Website links hosted on Weebly.com.

Major findings: are guided by the rubric supplied by the course instructor and are

detailed in the forms provided by the participants in the Appendix below.

EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
5
The primary reasons for conducting the study: engage peers in exploring the

information and usability of the multimedia projects developed by the four groups in

EDID 6508 Developing Instructional Media.

Description of the product and its context of use:

The story board for the multimedia project the Respiratory System was initially

developed by Sean Thomas, the groups multimedia engineer. The storyboard was

chosen by the other group members for the development of the project and was

supplemented with additional information by each group member, contributing

additional content, and relevant information to the development of the final multimedia

project presented for this evaluation.

The multimedia project is intended for upper elementary school students, and is a

tutorial based on the respiratory system. The program describes functions of, and

organs involved in respiration. The diseases that affect the respiratory system, things

you can do to maintain good respiratory health, and alternative remedies that can be

used to aid in the maintenance of a healthy respiratory system are covered in the lesson.

The usability goals

Goals identified for the product are outlined above under the section, Purpose of

the Study. In addition, the ability of peers to understand and analyze how artefacts and

information are gathered to build the e-learning project in order to engage the user are

examined.
Participants: Nyeisha George Minott and Stephanie Lionel.

How they were contacted: Participants were contacted by Whatsapp and Email
respectively.

Total number of participants: 2

EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
6

Appropriate demographic data: (2 females, age mid to late 30s)

Computer/web experience: Nyeisha has a degree in Computer Science, is the tech

member of group 2 and teaches Computer Science. Stephanie is an elementary school

teacher and is the tech specialist for group 1.

Occupation/major: Computer Science Teacher, Tech Specialist.

Methods:

Details of the evaluation method are outlined in the completed rating forms by both

participants in the Appendix.

Summary of Findings
Information about what is and is not working:
Both participants indicated a glitch in the program, and navigation difficulties

encountered at Question 1 of the multiple choice quiz.

Additional major feedback includes:
Nyeisha : absence of a help button , distraction to user when video opens in a new

window for viewing, addition of more narration and music.

Stephanie: inclusion of images for the symptoms of the diseases, re- positioning of

text and re-stating of some objectives.

The severity of the problems: The navigation problems will negatively impact the

users ability to access the multiple choice quiz to complete the tutorial.
Specific recommendations for improvement: The multimedia engineer has been

notified and the problems are currently not required to be addressed.

Screen shots of application: were taken to develop and produce the Voice Thread

presentation by Collaborative Group 3 to give an overview on the content of the

multimedia project.
Descriptive statistics of data collected for effectiveness and efficiency: please see

feedback on peers evaluation in the Appendix.

Satisfaction evaluation survey results: evaluations were fairly done.
Discussion on findings
EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
7
Subsequent to the distribution of the project for evaluation, it was discovered that the

version distributed was not the final one tested.

Conclusion

The major findings of the evaluation study of the program revealed errors and

glitches that caused navigation difficulties on the first multiple choice slide if the

answer given was incorrect. In addition,

Possible next steps for the design team: investigation into the findings, though there is

no current requirement to fix the errors, the team will meet for an after action review

of the operation of the multimedia project to rectify the problem.

Appendix: Multimedia Evaluation Reports from Nyeisha and Stephanie

Collaborative Group 3
EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
8
Please use the table below to indicate how you rate our multimedia project. The multimedia Rubric provided by the instructor
will guide your decision for ratings. Thank you for your constructive feedback.

Name: Nyeisha George-Minott Date 19/4/2017

Category Maximum Score Peer Reviewers Score Comments
Storyboard Planning Sheet 4 3.5 The multimedia project was
well thought out. The
storyboard matches with
what is seen in the actual
program to a large extent.
Organization of Content 4 3.5 There is a logical sequence
to your program, buttons for
back, forward and home.
Good addition of the topics
button
Originality 4 3 The program though similar
to others on the same topic
does have its own unique
characteristics. Missing
narrations for students who
may have difficulty reading
the slides.
Copyright and 4 2 I did not get to the end of the
documentation program to see a references
page.
Format and Platform 4 2.5 I used a PC to open the
Transferability program, cannot tell if there
are issues while using a
MAC.
Subject Knowledge 4 4 Very knowledgeable about
the topic. Especially
concerning the types of
diseases affecting the
respiratory system and the
parts
Graphic design 4 3 Text and images are
skillfully combined.
Mechanics 4 4 Found no issues here
Screen Design 4 3.5 Missing a button for help.
Users follow instructions
according to the narrator but
may need more guidance.
Use of Enhancements 4 3 Video content entices users
to learn, but can be a
distraction once users open a
new window to watch the
videos.

Total 40 32
EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
9
Additional Comments:

Upon opening the program does take while to load. I encountered a problem, I could not

get to the end. After the label the diagram activity, and moving on to the first question in the

multiple choice quiz, once I answered and I got it incorrect, I could not move forward. Clicking

submit takes me back to the labeling of the diagram. Clicking next after that returns to the

question with the same answer I submitted before, but cannot click any other options.

The automated voice of the character on screen was quite interesting. I like the addition

of the Topics button later in the program for easier navigation. Program could use some music in

the background and other sound effects where necessary.

Pertaining to the YouTube video links, the videos open in a separate window to watch the videos.

This could be a potential distraction as users may continue to watch YouTube videos and forget

to return to the multimedia lesson. Maybe embedding the video content can be an option.

Group Project Peer Evaluation Sheet

Collaborative Group 3
Please use the table below to indicate how you rate our multimedia project. The multimedia Rubric provided by the instructor
will guide your decision for ratings. Thank you for your constructive feedback.

Name: Stephanie Lionel Date: 22-04-17

Category Maximum Score Peer Reviewers Score Comments

Storyboard Planning 4 4 But can I share your
Sheet individual story board? I
am studying Caribbean
next week.
Organization of Content 4 4 Score in pre-test should
come above we have a
lot to learn
Originality 4 3.5 Selection of media is
unique but sequence and
EDID 6508 ASSIGNMENT 3: EVALUATING MULTIMEDIA PROJECT
10
structure.
Copyright and 4 4 Everything was cited
documentation
Format and Platform 4 3 It should work on another
Transferability computer or tablet. As it
is, my phone does not
have the memory or flash
needed to run this.
Subject Knowledge 4 4
Graphic design 4 4 What about images of the
symptoms of diseases?
Mechanics 4 3 Short song.
Objective 6, needs to be
stated as an objective.
Numbers need ot be more
specific (at least 1 or 2
instead of some
Screen Design 4 4
Use of Enhancements 4 3 I am assuming this is for
grades 4-6.

Total 40 36

Additional Comments:
1. Revise to add all End punctuation .
2. Not entirely necessary but faster readers should be able to skips sections
3. Click instruction should be in a new paragraph line. One different from information

line.
4. First songs chorus is catchy. Couldnt make out verse.accent.
5. Is there only one multiple choice question?
6. Like the experiment