Interpersonal Skills Training

TRAVAGLINI - FORMATIVE EVALUATION PLAN 1
Formative Evaluation Plan

Be the employee of the year: Hamford Universitys guide to mastering interpersonal skills in the
workplace
To: Learning Solutions Group at Hamford University
From: Cassandra Travaglini, student at Concordia MA Ed. Tech.
Date: Thursday, December 8, 2016
Subject: Plan for Conducting Formative Evaluations
Purpose of the evaluation

This plan presents a formative evaluation of the interpersonal skills course. This plan includes a
series of reviews in order to confirm that this program is accurate, is clear, and that it performs
properly. This plan begins with a technical review, then an editorial review, and finally, a
pilot/usability review.
Technical review
A technical review of the program follows each draft of the course except the final draft.
Participants
Review participants Anticipated expertise of participants
Psychology Professor at Area of research focuses on the study of personality and
Hamford University interpersonal relationships
Procedure for conducting the review

Advanced preparations in chronological Date Action
order with dates November 20, 2016 Create draft 1
December 1, 2016 Create draft 2
Materials for review (drafts, instructions & Material Delivery method
review criteria,) and methods of delivery Draft Email attachment
Instructions and Cover email
review criteria
Technical review Email attachment
report on draft
Review process Reading review to determine inaccuracies and
ambiguities in content as well as to verify
completeness in content.
Delivery date for drafts November 24, 2016
Required date for returns of drafts (at least November 30, 2016
1 day per 100pp of material)
Anticipated format for compiling review See Appendix: Technical review report
data, reporting and responding
Technical review meeting schedule Only one SME *Note: if no conflicting

remarks, meeting will be cancelled
Technical review meeting location Only one SME *Note: if no conflicting
remarks, meeting will be cancelled
Technical review meeting process Walkthrough, if required
Technical review meeting participants Participants whose remarks conflict
* Table formats adapted from David Prices Sample Formative Evaluation Plan 2010
See appendix for drafts of required documents.
Pilot/usability review
Usability testing follows the second draft of the course
Scheduling
Test Type Review date Review location
Preliminary with November 22, 2016 Concordia University
mockup
Final product To be determined To be determined
Participants
Test Type Review participants Anticipated expertise of participants
Preliminary with - Ilinca Enache Adult learners
mockup - Yang Gao
- Maryam Shoja
- Angie Argyrakos
Final product To be determined To be determined

Advanced preparations Date Action
in chronological order November 20, 2016 Complete draft of storyboard
with dates November 22, 2016 Recruit 4-10 learners
Set-up requirements for 1) Contact administration to reserve a seminar room, or a

usability test smaller classroom, at the University
2) Once the room is reserved, visit the room to make sure the
room has a projector and a cable to connect it to the laptop
used to store the course
3) Ensure that the room has speakers and test the speakers to
make sure they are connected and/or working properly
4) If speakers are not working, plan to bring speakers or ask
to rent speakers from the Universitys IT services
5) Make sure the room has a minimum of 10 seats and desks
available for the participants
6) Recruit between 8 and 10 learners/participants (including
an instructor to teach the course) and notify them of the
room number reservation and the time the pilot test will
take place
7) Plan to bring a clicker or borrow from the Universitys IT
service desk to allow for a smoother flow between slides
8) Plan to bring a pointer or borrow from the Universitys IT
service desk as it will be necessary to refer to a separate
image at a time on the slides
9) Save course document on both instructors laptop and USB
device
10)Download all video files included in the presentation onto
a USB, or the laptop you will be using, before the test
begins in order to make sure they are functioning and to
allow for the pilot to flow smoothly and be uninterrupted
by technical difficulties
11)Leave sufficient time to contact IT services if the videos do
not download or work properly.
12) If IT services are unable to find a solution, leave sufficient
time to review the production notes for the video scenarios
with the volunteer evaluators and plan a strategy to
perform the demonstrations in the video, in real-time in
front of the class instead.
13) Remind all participants 2-5 working days before the pilot
test
14) Practice conducting the course with instructor and
volunteer evaluators 2-5 days before the pilot test
15)Photocopies of each rubric provided at the end of the
instructors package will need tobe made in advance for
the potential 10 learners attending the pilot
16)Photocopies of each scenario and satisfaction survey
provided at the end of the instructors package will need to
be made in advance for the potential 10 learners attending
the pilot. The following pages must be distributed
separately (not all at once) to students based on the lecture
instructions provided in the instructor guide.
17)Photocopies of the package of slides for learners will need
to be made in advance for the potential 10 learners
attending the pilot
18)Bring a minimum of 10 pencils and loose leaf paper for
learners attending the pilot
19)Before the pilot begins, arrange tables and chairs to form
rows facing the screen if they are not already arranged in
this way
Beginning of the pilot 1) Distribute learners package of slides while a volunteer
evaluator debriefs the participants in order to be unbiased
2) In order to determine whether the course fits within the 30-

minute time frame, have the observer begin recording the
pilots starting time and instruct the observer to make sure
that the pilot continues to be recorded until its end time
3) Instruct the observer to write notes as learners ask
questions or make comments throughout the pilot
While the pilot is 1) Request feedback and suggestions on improvement after
conducted each unit and its assessment is completed (remind the
observer to stop the timer at this juncture and begin the
timer again after all feedback on the unit has been given)
2) Remind learners that the test is of the program and not
them
Debriefing at the end of 1) Ask participants about their overall impressions
the pilot 2) Ask participants about what is effective
3) Ask participants about what needs revision
4) Encourage participants to provide specific suggestions on
ways to fix the issues they have identified
5) Ask participants whether they plan to apply what they
learned
6) If participants do not plan to apply what they have learned,
ask them why and what would help them to do so
7) Debrief with the instructor to see if anything was unclear
and determine whether or not he or she felt comfortable
teaching the course or why he or
Assessing participants 1) Ask participants whether the assessment instructions were
performance on clear
assessment activities 2) Assess participants performance on the unit evaluations to
make sure that the questions truly address the objectives
3) Ask participants whether all instructions were clear
Categorizing comments 1) Comments will be categorized as A, B, or C
by importance 2) Feedback will be responded to in 2 business days
Anticipated format for See Appendix: Preliminary Usability test report
compiling usability
data, reporting and
responding
Production/Editorial review
The production/editorial review will occur once, before the course goes into final production.
Participants
Review participants Anticipated expertise of participants
To be determined Strong writer in English, able to resolve unclear passages and

enhance the presentation of the content so that learners will be able
to easily understand it

Advanced preparations in chronological Date Action
order with dates To be determined: Create final version
Will occur once the
issues involving the
usability review are
resolved
Materials for review (drafts, instructions & Material Delivery method
review criteria,) and methods of delivery Draft Email attachment
Instructions and Cover email
review criteria
Ed/prod review report Email attachment
on draft
Editorial and production guidelines See Appendix: Style guidelines
Review process Reading review to look at spelling, grammar,
punctuation, style, layout of slides as well as
to ensure:
1) Consistency of design across student
and instructor materials
2) Appropriate use of templates
3) Correct and consistent use of headings
4) Consistent style for charts
5) Proper placement of illustrations
6) Clearly marked margins
7) Correct and consistent use of fonts
Delivery date for drafts To be determined
Required date for returns of drafts (at least To be determined
1 day per 100pp of material)
Anticipated format for compiling review See Appendix: Production/editorial review
data, reporting and responding report
Review meeting schedule To be determined
Review meeting process Walkthrough, if required
Review meeting participants Participants whose remarks conflict
(author/editor)

Appendix
The following appendices were adapted from David Prices Sample Formative Evaluation
Plan 2010
Technical Review Documents
Advance notice of reviews

Hello,
I am conducting a technical review for a course called Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. The technical review
determines whether inaccuracies or ambiguities are present in the content.
You are one of the technical reviewers for the course based on your expertise in <EXPERTISE>.
I will be forwarding a draft of the course to you for technical review based on your area of
expertise.
I will be sending the draft on <SEND DATE>.
Please annotate the draft to identify and correct inaccuracies or ambiguities in content based on
your area of expertise. Please avoid comments regarding spelling, grammar or layout except
where it would affect the technical component of the content.
Please return the comments and the marked-up version of the draft to me by <RETURN DATE>.
Thank you for your participation. Your expertise is a valued part of course development.
Regards,
Cassandra Travaglini
Review instructions and criteria

Hello,
I am conducting a technical review for a course called Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. The technical review
determines whether inaccuracies or ambiguities are present in the content.
You are one of the technical reviewers for the course based on your expertise in <EXPERTISE>.
I will be forwarding a draft of the course to you for technical review based on your area of
expertise.
I have attached the draft of the course to this email for you to review.
Please return the comments and the marked-up version of the draft to me by <RETURN DATE>.
Regards,
Reminder letters
Hello,
You are a participant in a technical review for a course called Be the employee of the year:
Hamford Universitys guide to mastering interpersonal skills in the workplace.
I sent you a draft of the course on <SEND DATE>.
To continue with course development, I need to receive your comments by <RETURN DATE>.
Regards,
Announcements of the review meeting

Hello,
You are a participant in a technical review for a course called Be the employee of the year:
Hamford Universitys guide to mastering interpersonal skills in the workplace. The technical
review determines whether inaccuracies or ambiguities are present in the content.
Technical reviews may result in conflicting remarks. One expert may say something inaccurate
while another expert may insist it is correct. A review gathers the course developer and the
experts to discuss the conflicts and determine a solution.
I am scheduling a review meeting to take place on <REVIEW MEETING DATE>. Please set
this time aside on your calendar. In the event reviewers do not provide conflicting remarks, I will
cancel the meeting and notify you.
Regards,
Thank-you for feedback of participants

Hello,
I have received your feedback for a course called Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. Thank you for your
participation.
I will review your feedback and the feedback of the other participants in the review.
In the event reviewers provide conflicting remarks, we will resolve those conflicts during our
previously scheduled review meeting.
I will provide a report that outlines the feedback of reviewers and the solutions developed to
address those remarks.
Regards,
Report
Technical review report
Technical review for Be the employee of the year: Hamford Universitys guide to mastering
interpersonal skills in the workplace.
Course Developer: Cassandra Travaglini
Participants: Psychology Professor at Hamford University
Draft: November 20, 2016
Draft sent: November 24, 2016
Review meeting: Only one SME *Note: if no conflicting remarks, meeting would be cancelled
Overview
This report provides the remarks from a technical review for a course called Be the employee of
the year: Hamford Universitys guide to mastering interpersonal skills in the workplace. The
review determines inaccuracies or ambiguities in the course content.
This report begins by discussing the general remarks of the observer during the preliminary
usability test for the course. Below these remarks, the report lists the resolution determined by
the course developer. If the course developer chooses not to change content based on a remark,
the report provides the reason for this decision.
The report also includes remarks regarding specific visuals. Below the remarks, the report lists
the resolution determined by the course developer. If the course developer chooses not to change
content based on a remark, the report provides the reason for this decision.
Remarks on overall course
<OVERALL REMARKS>
<COURSE DEVELOPER RESOLUTION>
Remarks specific to visuals
<VISUAL REMARKS>
<COURSE DEVELOPER SOLUTION>
<REVISED VISUAL>
Usability Test Documents
Invitations to participate
Hello,
I am conducting a usability test for a course called Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. The usability test
determines whether the course is comprehensible and functional by running the course with trial
learners. Comprehensible means the learners understand the content and complete the
exercises without having to re-read the content or request additional help. Functional means
the course appears the way it is supposed to and videos and animations in the course operate as
necessitated.
The usability test does NOT test you as a learner. The purpose is only to test the usefulness of the
course.
I am inviting you to participate in the usability test. If you agree to participate, I will provide you
with the information you need to take the course.
The usability test will take place on <USABILITY DATE>.
During the usability test, you will take a 30-minute course as a regular student. An observer will
watch you and take notes. You will be encouraged to speak and ask questions during the course,
especially when you determine that you need clarification on the content presented to you. At the
end of the usability test, the observer will debrief you by asking for feedback on how
comprehensible and functional the course was for you.
Please let me know whether you can participate by <USABILITY RSVP DATE>.
Thank you for your participation. Your assistance is a valued part of course development.
Regards,
Storyboard usability report

Storyboard usability test report
Usability test for Be the employee of the year: Hamford Universitys guide to mastering
interpersonal skills in the workplace
Participants: Ilinca Enache, Yang Gao, Maryam Shoja, and Angie Argyrakos
Course draft: November 20, 2016
Usability test date: November 22, 2016
Overview
This report provides the results of a storyboard usability test for a course called Be the
employee of the year: Hamford Universitys guide to mastering interpersonal skills in the
workplace. The usability test determines whether the course is comprehensible and functional
by running the course with trial learners. Comprehensible means the learners understand the
content and complete the exercises without having to re-read the content or request additional
help. Functional means the course appears the way it is supposed to and videos and animations
in the course operate as necessitated.
Remarks rated according to their urgency:

A: Design and development should not continue before addressing the issues
B: Must change before the course is made available, although design and development
can continue
C: Would be ideal to change, though only if time permits
usability test for the course. Below the remarks, the report lists the resolution determined by the
course developer. If the course developer chooses not to change content based on a remark, the
report provides the reason for this decision.
One user determined that the courses use of grading rubrics is a very helpful and concrete way
of giving learners appropriate feedback that is directly related to the learning objectives. This
learner also I found that the courses evaluation methods are directly linked to the course
objectives and very accurately related to the courses practice opportunities.
Regarding practice opportunities, another user found it helpful to be provided with an assessment
problem immediately after each unit as learners will avoid becoming confused when learning the
different skills taught in each unit.
Users agreed that student instructions for the practice opportunities were unclear and they felt
unsure of what was expected for them. Instructor notes were therefore expanded on based on
these suggestions.
A few users mentioned that the use of videos throughout the course is a very effective way to
help learners identify and visualize appropriate interpersonal behaviours as this use adds an extra
touch to a plain written scenario because it is more closely connected to what learners will
witness in the real world. A few users also provided positive feedback for the simplicity and
clarity of the visuals and text on the slides.
One user determined that she was confused as to which unit or section she was in. The solution
she brought forth was to number the slides. For example, if the learner is in Unit 1, Part a., it
would be helpful to have those annotations written somewhere on the slide.
One user requested that the information in the visuals on slide 19 be broken up into two separate
slides. However, this was left unchanged and it was decided that the information was more
effective when it was presented together. The instructor would therefore be instructed to use his
or her laser pointer to refer to the subject on the slide that he or she was discussing in order to
help learners follow along
One user found that the metaphor concerning offering constructive criticism to build
relationships with colleagues on slide 6 was not descriptive of what offering constructive
criticism entails. She proposed to change the wording in the speech bubble to more accurately
reflect an example of what giving constructive criticism would sound like and the visual was
therefore adjusted accordingly.
Final reports
Usability test report
Usability test for Be the employee of the year: Hamford Universitys guide to mastering
Participants: <LIST PARTICIPANTS>
Course draft: <DRAFT NAME AND DATE>
Usability test date: <USABILITY DATE>
Overview
This report provides the results of a usability test for a course called Be the employee of the
year: Hamford Universitys guide to mastering interpersonal skills in the workplace. The
usability test determines whether the course is comprehensible and functional by running the
course with trial learners. Comprehensible means whether the learners understand the content
and complete the exercises without having to re-read the content or request additional assistance.
Functional means the course appears the way it is supposed to and videos and animations in
the course operate as necessitated.
Remarks rated according to their urgency:
A: Design and development should not continue before addressing the issues
B: Must change before the course is made available, although design and development
can continue
C: Would be ideal to change, though only if time permits
usability test for the course. Below the remarks, the report mentions the resolution determined by
the course developer. If the course developer chooses not to change content based on a remark,
the report provides the reason for this decision.
<OVERALL REMARKS>
<VISUAL REMARKS>
<REVISED VISUAL>
Production/Editorial Review Documents
Advance notice of reviews

Hello,
I am conducting a production/editorial review for a course called Be the employee of the year:
Hamford Universitys guide to mastering interpersonal skills in the workplace. The
production/editorial review identifies errors in spelling, grammar, style, and layout as well as any
inconsistencies in the structure of the content.
You are one of the production/editorial reviewers. I will be forwarding a draft of the course to
you for review.
I will be sending the draft on <P/E SEND DATE>.
Please annotate the draft and identify and correct errors in spelling, grammar, style, and layout of
the slides. Please also ensure:
1) Consistency of design across student and instructor materials

I have attached the style guide for Be the employee of the year: Hamford Universitys guide to
mastering interpersonal skills in the workplace to this email to assist you.
Please return the comments and the marked-up version of the draft to me by <P/E RETURN
DATE>.
Regards,
Style sheets
Style guide for Be the employee of the year: Hamford Universitys guide to mastering
Edition
Released December 1, 2016 by C. Travaglini
Overview
This style guide sets out the style to be used for Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. Consult this style guide
while drafting content for the course and while conducting a production/editorial review of the
course.
Style guidelines
Language: This course uses Canadian English. Please consult the Canadian Oxford Dictionary.
Voice: This course uses an active voice for all sentences (sentence subjects are doing the action
or being acted upon)
Layout: For slides other than introductions, contents, summaries, and assessments, this courses
uses the assertion-evidence layout. The assertion-evidence layout makes a statement as a
heading at the top of the page and presents a visual in the rest of the slide that provides evidence
or context for the statement.
Artwork and iconography: This course avoids using photographs and realistic artwork and uses
simple artwork to focus attention on important concepts. The artwork is presented using
metaphors that include figures of people with expressive body language. These figures are used
in all the introductory slides to each unit to engage learners and hold their attention. Charts are
used to in other slides that are denser in content.
Typeface and sizes on the content slides: Headings use 28pt Calibri. Body text uses 20-24pt
Calibri. For bulleted lists, headings use 24pt Calibri and body text uses 20pt Calibri.
Colours: This course uses a high contrast colour scheme of white backgrounds and black text.
Visuals use blue to emphasis a neutral state not charged by emotion.
Reminder letters
Hello,
You are a participant in a production/editorial review for a course called Be the employee of the
year: Hamford Universitys guide to mastering interpersonal skills in the workplace. I sent you
a draft of the course on <SEND DATE>.
To proceed with course development, I need your comments by <RETURN DATE>.
Please annotate the draft and identify and correct errors in spelling, grammar, style, and layout of
slides. Please also ensure:
1) Consistency of design across student and instructor materials

Please use the style guide for Be the employee of the year: Hamford Universitys guide to
mastering interpersonal skills in the workplace which I provided to your earlier.
Regards,
Announcements of the review meeting

Hello,
You are a participant in a production/editorial review for a course called Be the employee of the
year: Hamford Universitys guide to mastering interpersonal skills in the workplace. The
production/editorial review identifies errors in spelling, grammar, style, and layout as well as any
inconsistencies in the structure of the content.
Production/editorial reviews may result in conflicting remarks. One expert may say something
inaccurate while another expert may insist it is correct. A review gathers the course developer
and the experts to discuss the conflicts and determine a solution.
I am scheduling a review meeting to take place on <REVIEW MEETING DATE>. Please set
this time aside on your calendar. In the event reviewers do not provide conflicting remarks, I will
cancel the meeting and notify you.
Thank you for your participation. Your expertise is a valued part of course development
Regards,
Thank-you for feedback of participants

Hello,
I have received your feedback for a course called Be the employee of the year: Hamford
Universitys guide to mastering interpersonal skills in the workplace. Thank you for your
participation.
I will review your feedback and the feedback of the other participants in the review.
In the event reviewers provide conflicting remarks, we will resolve those conflicts during our
previously scheduled review meeting.
I will provide a report that outlines the feedback of reviewers as well as the solutions developed
to address those remarks.
Regards,
Draft of the report

Production/editorial review report
Production/editorial review for Be the employee of the year: Hamford Universitys guide to
mastering interpersonal skills in the workplace
Participants: <LIST PARTICIPANTS>
Draft: <DRAFT NAME AND DATE>

Draft sent: <SEND DATE>
Review meeting: <REVIEW MEETING DATE>
Overview
This report provides the results of a production/editorial review for a course called Be the
employee of the year: Hamford Universitys guide to mastering interpersonal skills in the
workplace. The production/editorial review identifies errors in spelling, grammar, style and
layout. The review also identifies inconsistencies in the structure of the content.
This report begins by discussing the general remarks of reviewers including any resolutions of
conflicts through review meetings. Below the remarks, the report lists the resolution determined
by the course developer. If the course developer chooses not to change content based on a
remark, the report provides the reason for this decision.
The report also discusses each visual in the course that received remarks and lists the relevant
remarks below the visual including any resolutions of conflicts through review meetings. Below
the remarks, the report lists the resolution determined by the course developer concerning the
updated visual. If the course developer chooses not to changed content based on a remark, the
report provides the reason for this decision.
<OVERALL REMARKS>
<VISUAL REMARKS>
<COURSE DEVELOPER SOLUTION>
<REVISED VISUAL>
Acknowledgments
The outline and appendices for this formative evaluation were adapted from Sample Formative
Evaluation Plan by David Price provided as part of the Fundamentals of Instructional Design
ETEC 650 class
Carliner, S. (2015). The basic quality checks for training programs. In Training Design Basics
(2nd ed., pp. 175-190). Alexandria, VA: ATD Press.

Interpersonal Skills Training

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Interpersonal Skills Training

Uploaded by

Copyright:

Available Formats

TRAVAGLINI - FORMATIVE EVALUATION PLAN 1

Formative Evaluation Plan

To: Learning Solutions Group at Hamford University

From: Cassandra Travaglini, student at Concordia MA Ed. Tech.

Date: Thursday, December 8, 2016

Subject: Plan for Conducting Formative Evaluations

Purpose of the evaluation

Procedure for conducting the review

Technical review meeting schedule Only one SME *Note: if no conflicting

See appendix for drafts of required documents.

Usability testing follows the second draft of the course

Procedure for conducting the review

Set-up requirements for 1) Contact administration to reserve a seminar room, or a

2) In order to determine whether the course fits within the 30-

See appendix for drafts of required documents.

To be determined Strong writer in English, able to resolve unclear passages and

Procedure for conducting the review

See appendix for drafts of required documents.

Technical Review Documents

Advance notice of reviews

I will be sending the draft on <SEND DATE>.

Review instructions and criteria

I sent you a draft of the course on <SEND DATE>.

Announcements of the review meeting

Thank-you for feedback of participants

Course Developer: Cassandra Travaglini

Participants: Psychology Professor at Hamford University

Draft: November 20, 2016

Draft sent: November 24, 2016

Remarks on overall course

<COURSE DEVELOPER RESOLUTION>

Remarks specific to visuals

<COURSE DEVELOPER SOLUTION>

Usability Test Documents

The usability test will take place on <USABILITY DATE>.

Storyboard usability report

Course Developer: Cassandra Travaglini

Course draft: November 20, 2016

Usability test date: November 22, 2016

Remarks rated according to their urgency:

C: Would be ideal to change, though only if time permits

Remarks on overall course

Remarks specific to visuals

Course Developer: Cassandra Travaglini

Participants: <LIST PARTICIPANTS>

Course draft: <DRAFT NAME AND DATE>

Usability test date: <USABILITY DATE>

Remarks rated according to their urgency:

C: Would be ideal to change, though only if time permits

Remarks on overall course

<COURSE DEVELOPER RESOLUTION>

Remarks specific to visuals

<COURSE DEVELOPER RESOLUTION>

Production/Editorial Review Documents

Advance notice of reviews

I will be sending the draft on <P/E SEND DATE>.

1) Consistency of design across student and instructor materials

7) Correct and consistent use of fonts

Released December 1, 2016 by C. Travaglini

To proceed with course development, I need your comments by <RETURN DATE>.

1) Consistency of design across student and instructor materials

Announcements of the review meeting

Thank-you for feedback of participants

Draft of the report