Professional Documents
Culture Documents
by
Cybele Wu
IDT 8001, CRN 23851
Instructional Design
3
on the one-on-one evaluation phase of the formative evaluation plan.
The Subject Matter Expert (SME) was contacted to evaluate the
instructional materials. He was contacted via email to review the
materials as well as the interview questions. Another in-person
interview occurred to go over the instructional materials. The interview
was recorded to make the changes to the instructional materials using
the
this project. In this phase of the project, the SME was contacted via
email to review the instructional materials. In the email (Appendix C),
the Subject Matter Expert Protocol (Appendix B), along with the
instructional materials the
Procedure
This learner read the learner evaluation protocol (Appendix E)
prior to going through the print-based procedure, as laid out in the
lesson. They completed the observer checklist (Appendix G) to ensure
that they completed all steps in order. Once they completed the
checklist, they completed the lesson evaluation form (Appendix H) to
determine if all goals and objectives were completed. They made a
note of what aspects should be reassessed.
Second Learner Review
Subjects
The second learner was a learner with intermediate technical
experience. This learner was a fellow classmate who played the part of
an intermediate learner. This learner was comfortable with basic video
recording software, as well as basic computer functions. This learner
has played around with command prompts but was not completely
comfortable with them. This learner represents the intermediate level
of technological abilities concerning the target audience. The learner
completed the print-based instruction on their fully accessible
computers. Instruments
The instructional designer used the evaluation protocol to
describe the evaluation process to the learner. The learner checklist
(Appendix G) was used to determine if all steps in the proper order
were completed for the lesson. It was given to the learner along with
all instructional materials. Finally, after the lesson was completed, the
learner used the lesson evaluation form (Appendix H)
to indicate what changes (if any) should be made clearer in the lesson
content, goals, and objectives.
Procedure
This learner read the learner evaluation protocol (Appendix E)
prior to going through the print-based procedure, as laid out in the
lesson. They completed the learner checklist (Appendix G) to ensure
that they completed all steps in order. After they completed the
checklist, they completed the lesson evaluation form (Appendix H) to
determine if all goals and objectives were completed. They made note
of what aspects should be reassessed.
Results
The SME had one week to evaluate and review the instructional
materials. The table below summaries his comments and where they
should be modified.
Instructional
SME Comments
Materials
3.3 Using the
command to save
video in Web-M, VP-9
format
Learner Comments
Instructional
Beginner Comments
Materials
Instructional
Materials meeting
criteria and
objectives of the
lesson.
Directions Clear and
easy to understand.
Images helpful in
completing lesson.
The steps for lesson
were given in a
logical order.
The layout of the
instructional
materials had an
appealing look.
The instruction
provided was
adequate to
compress highdefinition video to
similar quality in a
smaller file size in
Web-M, VP-9 format
for uploading and
sharing.
The instructional
materials, including
the text and
images, motivated
me throughout the
Intermediate
Comments
No Comment
Some problems
with copying
command that it
adds extra spaces
Instructions and
Images separated
on two pages one
time.
No Comment
No Comment
No Comment
No Comment
lesson.
The lesson can be
completed in less
than thirty minutes.
I feel my knowledge
is proficient enough
to complete this
lesson again using
the instructional
materials.
The instructional
materials will be
used again to
compress highdefinition videos to
Web-M, VP-9 format
for sharing and
uploading.
No Comment
Discussion
When analyzing the results from the SME and the two learners, it
was made clear that the instructions did not give instructions for both
operating systems: PC and Mac. Although the beginner learner did not
indicate this in the lesson evaluation form (Appendix L), she had said
that her machine was only a Windows with 64-bit. It is unclear at this
point whether her problem with downloading was due to this, or if there
was another reason. Although, it does appear so, as the intermediate
learner was able to download on both a PC and a Mac.
Recommendations
As with many drafts, they are not without problems. In this case,
the main issue had to do with using different operating systems. With
different operating systems, the steps might be slightly different. For
example, for the beginner learner, she stated in step 1.6 something was
missing such as the extraction of .exe file. This would only be for a
Windows operating system, not a Mac. The command code is also a little
different, omitting ./ and extra spaces for Windows machines.
When writing the instructions, they were only written for the Mac
operating system, not Windows. Although this lesson works on all
operating systems: Windows, Mac, and PC, it is suggested that
instructions be written for all operating systems and be tested on each.
The other option is to require that learners operate this lesson on one
type of operating system and only write instructions for that operating
system. Although limiting it to one operating system would be more
concise and clearer, not all ESL instructors would have access to a Mac.
Therefore, it is suggested that instructions include instructions for at a
minimum PC and Mac, since the majority of people own either a PC or a
Mac.
References
Dick, W., Carey, L., & Carey, J. O. (2015). The systematic design of
instruction (8th ed.). Upper Saddle River, NJ: Pearson.
Appendix
A Overall
Plan
Phase
Exper
t
Revie
ws
One-toOne
Evaluatio
ns
Evaluator
Subject
Matter
Expert
Learner
1:
Beginner
Learner 2:
Intermedia
te
Date
Target
st
December 1 Goals,
Objectives,
&
Lesson
Due by
Content,
December
5th
Assessme
nt
th
December 5 Lesson
Content,
Assessme
nt
Instrument
s
Interview
Observer
Checklist
, Lesson
Evaluatio
n
Observer
Checklist
, Lesson
Evaluatio
n
APPENDIX
B
SUBJECT MATTER EXPERT FORMATIVE EVALUATION
PROTOCOL
The Subject Matter Expert will receive the directions as well as the instructional
materials via email. The Subject Matter Expert will participate in the formative
evaluation phase of the design process. He will be given these instructions, along with the
instructional materials. The SME will review the instructional materials prior to meeting
with the instructional designer. At the meeting, the SME will be asked questions to
determine whether the instructional goal: using pre-recorded high-definition video,
students will compress the video in Web-M, VP-9 format, on a computer with full
security allowances, while maintaining similar quality produced in a smaller size and
upload time is met through the instructional materials. The questions will be recorded to
have a record of what changes should be made. The feedback from the SME from this
interview will determine what changes, if any, should be made to make the content,
goals, & objectives. The design of the instruction will be changed based on the feedback
from this interview, prior to giving learners access to the materials.
APPENDIX C
SUBJECT EVALUATION LETTER:
Dear Subject Matter Expert:
Thank you for participating in the project. You have been chosen because of your
expert knowledge in video compression software. The goal of this project is to instruct
higher-Ed instructors of English as a Second Language on how to use the compression
software with a high-definition video so that it is compressed into web-m, VP-9 format,
and maintaining similar quality in a smaller file size that can be uploaded for sharing.
Please review the enclosed instructional materials along with the protocol. We will me
st
meeting on December 1 to go over the instructional materials, where you will be asked a
series of questions to determine the quality and focus of the design materials. Through
these questions, we will determine what changes, if any, should be made in the design of
the instructional materials.
I appreciate the time and knowledge you have contributed to this project
Cybele Wu
Instructional Designer
APPENDIX D
SUBJECT MATTER EXPERT INTERVIEW
QUESTIONS
1. The target audience is ESL higher-Ed instructors. Do you think that the objectives
in the materials are applicable for this target audience? Should they be adjusted? If
so, how?
2. Are the skills that are needed to perform this lesson identified? Are there any
skills that should be added or removed?
3. Do the steps for the instructional materials make sense? If not, how should they
be organized?
4. Is the instructional material fitting for the target audience? If not, what aspect of
the materials should be changed to fit the target audience?
5. Is the vocabulary in the instructional materials clear and easy to understand? If
not, how would your reword them?
6. Are the images present in the instructional materials easy to understand for
learners to complete the lesson? If not, what images should be included or
removed to strengthen lesson?
7. Is the learner checklist form and lesson evaluation form applicable for assessing
the learners performance of the lesson? If not, how would you change the
assessment so that it better assesses the learning outcome?
8. Do you have any comments, questions, or suggestions for either the design or the
instructional materials?
APPENDIX E
LEARNER FORMATIVE EVALUATION
PROTOCOL
After the Subject Matter Expert (SME) has reviewed the instructional materials,
the learners will be sent the learner evaluation protocol, instructional materials, checklist,
and lesson evaluation form.
Each learner will complete all steps in order of the print-based lesson. They will
complete the checklist to ensure they have completed all steps in the instructional
materials. Once the lesson has been completed, they will complete the lesson evaluation
form. The information on this form will help the instructional designer better redesign the
instruction so that all instructional goals and objectives are met.
Dear Learner:
A
P
P
ENDIX F
LEARNER EVALUATION
LETTER
Thank you for participating in this project. For this project, you will be
playing the part of an ESL (English as a Second Language) Higher-Ed
instructor. The goal of this lesson is to use open source compression
software to compress videos to Web-M, VP-9 format to maintain
similar quality in a smaller file size for faster uploading speed.
In this project, you will first read the learner protocol, use the
checklist to ensure all items are completed in order. Finally, you will
complete the lesson evaluation. The purpose is to provide feedback
on the lesson design and format to ensure that areas in question are
redesigned so that the goals and objectives are met.
I have enclosed the instructional materials, learner protocol,
checklist, and lesson evaluation. Please complete all sections and
send back to me by December 5 th. Thank you again for your
participation.
Cybele Wu
Instructional Designer
APPENDIX G
LEARNER
CHECKLIST
Checklist for Entry Skills
1.1.1 Learner is able to move around web page in 30 seconds or less.
1.1.1 Learner is able to move their mouse over an image to bring up options
1.6.1 Learner is able to install software correctly.
2.1.1 Learner is able to record a video with their cellphone.
2.3.1 Learner is able to copy files to their computer.
2.3.2 Learner is able to name files on the computer.
3.1.1 Learner is able to use command prompts to locate files.
3.2.1 Learner is able to find where file has been saved.
3.3.1 Learner is able to save file in a different format.
4.3.1 Learner is able to post on social media sites (Facebook, YouTube, Vimeo,
etc.,).for Subordinate Skills for Installing Video Transcoding Software
Checklist
1.1 Learner is able to open a browser (Safari, Internet Explorer, or Chrome)
http://www.ffmpeg.org/download.html.
1.3 Learner is able to access the "Get Packages" section of the web page.
1.4 Learner is able to hover the mouse over the appropriate image for
their operating system.
1.5 Learner has been successful in downloading software.
1.6 Learner has installed software successfully.
1.7 Learner has rebooted computer.
Checklist for Subordinate Skills for Recording 1-minute video (2.0)
(*Note if Video is Pre- recorded go to 3.0)
2.1 Learner has recorded a 1-minute video.
2.2 Learner has connected cellphone to computer via USB cable.
2.3 Learner has copied video recorded onto their computer.
Checklist for Subordinate Skills for Transcoding Video into Web-m, VP9
(3.0) has been successful in bringing up command prompt or terminal
3.1 Learner
format
3.2 Learner has located the file saved using the command prompt.
window
3.3 Learner has saved file as Web-m, VP9 format using the command prompt.
Checklist for Subordinate Skills for Uploading Transcoded video onto
a video-sharing website (4.0)
4.1 Learner has accessed video-sharing website.
4.2 Learner has located the file in Web-m format.
4.3 Learner has successfully uploaded video file in Web-m, VP9 format
APPENDIX H
LESSON EVALUATION
FORM
Yes No Comments
1. The instructional materials met
the criteria of the objectives for
this lesson.
2. The directions are clear and
easy to understand.
3. The images included in the
instructional material were
helpful in finishing the lesson.
4. The steps in the instructional
material were given in an
obvious order.
5. The layout of the
instructional materials had
an appealing
6. The
instructionlook.
provided was
adequate to compress highdefinition video to similar
quality in a smaller file size in
Web-M, VP-9 format for
uploading
and sharing.
7. The
instructional
materials,
including the text and images,
motivated me throughout the
lesson.
8. The
lesson can be completed in
less than thirty minutes.
9. I feel my knowledge is
proficient enough to complete
this lesson again using the
10.instructional
I will use thematerials.
instructional
materials again to compress
high-definition videos to WebM, VP-9 format for sharing and
uploading.
APPENDIX I
SME COMPLETED INTERVIEW QUESTIONS
1. The target audience is ESL higher-Ed instructors. Do you think that the
objectives in the materials are applicable for this target audience? Should they be
adjusted? If so, how?
Yes,Idoindeedthinktheobjectivesofthesematerialsareapplicableforthistarget
audience.Forexample,IhaveafriendwhowasdoingremoteESLtrainingof
peopleinUkraineusinglivevideoonGoogleHangoutsfromanorganicfarmin
Blacksburg,VA.SoIcouldcertainlyseeESLinstructorsusingvideothatthey
uploadtovariousonlineplatforms.Ithinkthematerialsareverygood,asidefroma
couplesmallcorrectionsthatIpointoutbelow.
2. Are the skills that are needed to perform this lesson identified? Are there
any skills that should be added or removed?
Yes,theskillsthatareneededareidentified.
3. Do the steps for the instructional materials make sense? If not, how should
they be organized?
Yes,thestepsmakesense.They'reorganizedwell,inchronologicalorder.
4. Is the instructional material fitting for the target audience? If not, what aspect
of the materials should be changed to fit the target audience?
Yes,theinstructionalmaterialsarefittingforthetargetaudience.
7. Is the learner checklist form and lesson evaluation form applicable for
assessing the learners performance of the lesson? If not, how would you
change the assessment so that it better assesses the learning outcome?
Yes,thelearnerchecklistformandlessonevaluationformareapplicablefor
assessingthelearner'sperformanceofthelesson.
8. Do you have any comments, questions, or suggestions for either the design or
the instructional materials?
ItistruethatonWindowscomputers,the"./"isnotneededinfrontofthe
commandonthecommandline.Theinstructionsshouldbeamendedtoaddthat
informationintostep3.3.
APPENDIX J
COMPLETED CHECKLIST BEGINNER LEARNER
Checklist for Entry Skills
x 1.1.1 Learner is able to move around web page in 30 seconds or less.
x 1.1.1 Learner is able to move their mouse over an image to bring up options
1.6.1 Learner is able to install software correctly.
2.1.1 Learner is able to record a video with their cellphone.
2.3.1 Learner is able to copy files to their computer.
2.3.2 Learner is able to name files on the computer.
3.1.1 Learner is able to use command prompts to locate files.
3.2.1 Learner is able to find where file has been saved.
3.3.1 Learner is able to save file in a different format.
4.3.1 Learner is able to post on social media sites (Facebook, YouTube, Vimeo,
etc.,).for Subordinate Skills for Installing Video Transcoding Software
Checklist
1.1 Learner is able to open a browser (Safari, Internet Explorer, or Chrome)
http://www.ffmpeg.org/download.html.
1.3 Learner is able to access the "Get Packages" section of the web page.
1.4 Learner is able to hover the mouse over the appropriate image for
their operating system.
1.5 Learner has been successful in downloading software.
1.6 Learner has installed software successfully.
1.7 Learner has rebooted computer.
Checklist for Subordinate Skills for Recording 1-minute video (2.0)
(*Note if Video is Pre- recorded go to 3.0)
2.1 Learner has recorded a 1-minute video.
2.2 Learner has connected cellphone to computer via USB cable.
2.3 Learner has copied video recorded onto their computer.
Checklist for Subordinate Skills for Transcoding Video into Web-m, VP9
(3.0) has been successful in bringing up command prompt or terminal
3.1 Learner
format
window
3.2 Learner has located the file saved using the command prompt.
3.3 Learner has saved file as Web-m, VP9 format using the command prompt.
Checklist for Subordinate Skills for Uploading Transcoded video onto
a video-sharing website (4.0)
4.1 Learner has accessed video-sharing website.
4.2 Learner has located the file in Web-m format.
4.3 Learner has successfully uploaded video file in Web-m, VP9 format
APPENDIX K
COMPLETED CHECKLIST INTERMEDIATE LEARNER
Checklist for Entry Skills
x 1.1.1 Learner is able to move around web page in 30 seconds or less.
x 1.1.1 Learner is able to move their mouse over an image to bring up options
x 1.6.1 Learner is able to install software correctly.
x 2.1.1 Learner is able to record a video with their cellphone.
x 2.3.1 Learner is able to copy files to their computer.
x 2.3.2 Learner is able to name files on the computer.
x 3.1.1 Learner is able to use command prompts to locate files.
x 3.2.1 Learner is able to find where file has been saved.
x 3.3.1 Learner is able to save file in a different format.
x 4.3.1 Learner is able to post on social media sites (Facebook, YouTube, Vimeo,
etc.,).for Subordinate Skills for Installing Video Transcoding Software
Checklist
x 1.1 Learner is able to open a browser (Safari, Internet Explorer, or Chrome)
x http://www.ffmpeg.org/download.html.
x 1.3 Learner is able to access the "Get Packages" section of the web page.
x 1.4 Learner is able to hover the mouse over the appropriate image for
their operating system.
x 1.5 Learner has been successful in downloading software.
x 1.6 Learner has installed software successfully.
x 1.7 Learner has rebooted computer.
Checklist for Subordinate Skills for Recording 1-minute video (2.0)
(*Note if Video is Pre- recorded go to 3.0)
x 2.1 Learner has recorded a 1-minute video.
x 2.2 Learner has connected cellphone to computer via USB cable.
x 2.3 Learner has copied video recorded onto their computer.
Checklist for Subordinate Skills for Transcoding Video into Web-m, VP9
(3.0) has been successful in bringing up command prompt or terminal
3.1 Learner
x format
3.2 Learner has located the file saved using the command prompt.
x window
x 3.3 Learner has saved file as Web-m, VP9 format using the command prompt.
Checklist for Subordinate Skills for Uploading Transcoded video onto
a video-sharing website (4.0)
x 4.1 Learner has accessed video-sharing website.
x 4.2 Learner has located the file in Web-m format.
4.3 Learner has successfully uploaded video file in Web-m, VP9 format
APPENDIX L
COMPLETED LESSON EVALUATION FORM BEGINNER LEARNER
Yes No Comments
1. The instructional materials met
the criteria of the objectives for
this lesson.
x
x
x
x
See comments @ #1,2,3. I
am not sure what
happened. I think it would
be a very useful lesson.
x
x
x
x
APPENDIX M
COMPLETED LESSON EVALUATION FORM INTERMEDIATE
LEARNER
Yes No Comments
1. The instructional materials met x
the criteria of the objectives for
this lesson.
x
2. The directions are clear and
easy to understand.
3. The images included in the
instructional material were
helpful in finishing the lesson.
4. The steps in the instructional
material were given in an
obvious order.
5. The layout of the
instructional materials had
an appealing look.
6. The instruction provided was
adequate to compress highdefinition video to similar
quality in a smaller file size in
Web-M, VP-9 format for
uploading
and sharing.
7. The
instructional
materials,
including the text and images,
motivated me throughout the
lesson.
8. The
lesson can be completed in
less than thirty minutes.
9. I feel my knowledge is
proficient enough to complete
this lesson again using the
10.instructional
I will use thematerials.
instructional
materials again to compress
high-definition videos to WebM, VP-9 format for sharing and
uploading.
x
x
x
x
x
x