Professional Documents
Culture Documents
TABLE OF CONTENTS
Page
I. Introduction.... 4
II. Methodology.... 5
a. Prototype ............ 5
b. Learners .................
17
18
Introduction
PowToon is a software for creating animated presentations with entertaining characters,
objects, sounds, and effects. In the past, professionals presenting to an audience have been
restricted by specific software with limited capabilities. For those in need to produce a high
amount of presentations on a regular basis, there was a shortage of resources and creativity.
PowToon provides options, tools, and creative outlets to craft videos or presentations which will
keep the audience engaged.
The instructional module presented focuses on giving the learner reliable guidance on
how to use PowToon. It is prepared with interactive sections to maintain the learner involved and
captive throughout the lesson. Our goal was to observe a change in responses of participants that
would become knowledgeable and skilled to use this software in the future. According to
Margaret E. Gredler in Learning and Instruction: Theory into Practice: the evidence that
learning has occurred is an observed change in behavior, typically based on a comparison of preand post-instructional performance. (p.145). In her discussion of Ganges Concepts of
Capabilities, she argues that learning results in humans acquiring skills, knowledge, attitudes,
and values. This study will provide the scientific evidence to show an observed change in
participants skills and knowledge while creating videos and presentations by using PowToon.
We predict in this study that participants will score higher on the post-test after viewing our
instructional module on how to use PowToon. They will be able to acquire skills that will help
them create a video or presentation based on the steps presented in the instruction.
Methodology
Prototype
The lesson prototype that was chosen for this study is a screen capture designed to show
the steps from start to end on how to create a simple PowToon video or presentation. When we
decided to use PowToon as our mode of instruction, we recognized that the video alone would
not suffice. For this reason, we chose Zaption. Zaption is a video-based program that provides
tools and interactions that can be added to any video. It provides comprehension checks
throughout the video itself that allow for a more engaging instructional experience for the
learner. For example, learners have to answer multiple choice questions about signing in options
into PowToon, choose a true or false statement describing a function of PowToon, or click on all
related answers to a question linked to functionality in the program. Consequently, with this tool
we were able to create a better instructional tool to address our objectives in a direct manner.
Learners
The target audience for this project consisted of participants with different professional
backgrounds. We chose individuals who possessed the necessary factors for learning (Stolovitch
and Keeps, 2011, p.40). In their book: Telling Aint Training Stolovitch and Keeps presented
key ingredients for learning: ability, prior knowledge, and motivation. Participants in this study
had the ability to learn, prior knowledge of using technology (computers, emails, presentations),
and motivation to help in our study and learn a new skill. They were familiar with creating noninteractive presentations and did not need any prerequisite skills to learn PowToon. We
recognized that learning how to create an interactive presentation with animation can be a useful
tool at their work. Participants will acquire the necessary skills needed by watching the
instructional module provided.
Tryout Process
Pre and Post-Test. Our initial contact with participants was an invitation sent by email
or Facebook Messenger that included details of our project (Appendix A). Individuals that
expressed interest in being part of the activity were then contacted with a link to the pre-test
(Appendix B). Both pre- and post- tests were created using Google Forms. Our invitation was
easy to understand, and it listed a short explanation and the steps required by participants. We
divided the steps to part one, the pre-test. Once the pre-test was completed, part two was sent
(Appendix C). Part two contained the remainder of the links. It included the links to the
instructional module, the post-test, and the usability test (Appendix E). Additionally, the steps
provided approximate time participants would need to complete each step. This process was
intended to help them determine if they can invest the time to complete the process.
Observation. Before presenting the prototype to our target audience, we ran the initial
product sample by a younger audience where we were able to observe in person the reactions of
the participants and determine the ease of use. Based on this unstructured observation, we made
a few adjustments to the prototype. For example, we added more interactive sections in the video
to avoid long segments with no reciprocal action. Additionally, we replaced some notes with
questions and in some cases, we changed the type of questions. Our observation was to collect
some exploratory data for improvements on the product. The subjects were not aware of the
observers task; the environment was kept natural and informal.
Questionnaire. The questionnaire was also created using Google Forms. The questions
were based on a five-point Likert Scale. Participants were asked various questions about their
experience through the instructional module and their experience as a whole. When creating the
questionnaires, we took into consideration the guidelines and rules discussed in Evaluation in
Organizations: A Systematic Approach to Enhancing Learning, Performance, and Change by
Russ-Eft, D., & Preskill, H. (2009) to ensure quality and clarity of survey. For example, we
avoided double-barreled questions, leading and loaded questions, and negatives. We made sure
our questions had a single purpose and were simple, clear, and short.
Tryout Conditions
We contacted participants by email and Facebook Messenger. The tryout conditions were
expected to be favoring a complete response in a short amount of time. Because of logistical
reasons we decided to complete our project remotely. We also viewed it as a positive to have
participants complete the task in the comfort of their home or environment of their choice. It
would also allow flexibility in terms of time and schedule.
Results
Entry Conditions
Participants taking the test have no prior experience using PowToon. All 11 users are
computer literate and can access the internet. The initial invitation to participate was sent via
Facebook Messenger or personal email. The link included an invitation to participate (Appendix-). Initially, the request was sent to approximately 20 users. Participants reported difficulties with
viewing the instructional module on their SmartPhone. A specific Zaption application would
have to be downloaded to view our project on a phone versus a desktop/laptop computer. There
were no other unusual entry conditions observed.
Instruction
This module was designed to be completed remotely. It is an interactive and self-paced
instruction designed to be used by participants at a place of their choice and convenient time.
The only possible comment that can be made on observed behavior would be in our practice trial
with the younger audience. We anticipated the need for interactivity and discovered more was
required after these observations. One participant had to be dropped because she completed the
pre-test but was unable to complete the rest of the project, which brought our participant number
from 12 to 11.
According to the usability report (Appendix F), 10 out of 11 users agreed that the
instruction was clear. 9 out of 11 agreed that the lesson informational structure is appealing. 10
out of 11 agreed that they will use PowToon as a tool if needed in the future.
Outcomes
The analysis of data retrieved from pre- and post-tests indicated that participants reached
a significant learning outcome as a result of viewing the module on How to learn PowToon.
This data concurs with our initial prediction of possible outcomes. Both pre- and post-tests were
based on dependent samples as they represent an example of a repeated measure. To
demonstrate: we conducted a pre-test before participants viewing the instructional module. Then
we conducted a post-test. Therefore, we chose the t-test: paired two sample for means option.
The null hypothesis is a directional hypothesis: H0: 1<=2, which also means that the research
hypothesis is directional as well 1<2. Based on the numbers from our research data, we are
comparing the absolute value of the t-Stat with one-tailed critical value. (Table1) The t-Stat
result is bigger than the critical value (t=-5.78 > 1.81). Therefore, we reject the null hypothesis
and support the research hypothesis. Thus confirming participants performed better on the posttest after viewing the instructional module on How to use PowToon.
Information collected using the pre-test provided additional information on the
participants. There were ten females and one male that completed all the parts of the module.
Our sample was not a random sample of contributors; it was a convenient sample due to time and
space constraints.
Table 1
t-Test: Paired Two Sample for Means
Pre-Test
Mean
Variance
Observations
Post-Test
14
4
11
Pearson Correlation
0.2494337479
Hypothesized Mean
Difference
df
t Stat
P(T<=t) one-tail
t Critical one-tail
P(T<=t) two-tail
t Critical two-tail
10
-5.776554342
0.00008928603116
1.812461123
0.0001785720623
2.228138852
18.27272727
4.018181818
11
10
Table 2
Descriptive Statistics
Alpha (for confidence interval)
.5%
Pre-test Results
Post-test Results
Count
11
11
Mean
14.
18.2727272727
Variance
4.
4.0181818182
Standard Deviation
2.
2.004540301
0.1428571429
0.1097012105
Minimum
9.
14.
Maximum
16.
20.
Range
7.
6.
Median
15.
19.
13.5
18.
15.
19.5
Sum
154.
201.
Mode
15.
19.
Coefficient of Variation
Recommendations
Based on the pre- and post-tests, instructional module, and its usability participants gave the
following feedback as short answers. One participant emailed additional comments that are included
in this list.
Pre-set question allows true and false to both be selected (checkboxes are used instead of
radio button)
11
12
have worked properly during the participant's experience. Regardless of experience, these
comments are valuable for future implementation of the module.
A Timeline is: question, the option #2 should read: The section in the project (bar)
where you can manage the text display duration and when the time at which it will appear
within the slideshow.
Scrubbing is quickly viewing the progress of a slide by moving the playhead manually.
This should be reworded to include moving the playhead forward or backward within the
slide.
Needs an attention grabber and screen was too small which may lose some interest.
I think the breaks for questioning were important but perhaps too many. So it made it
very difficult to pay attention to the graphics and also the content. The questions made it
difficult to absorb the information or focus on items of personal interest.
I wish it was a little bit slower.
Sending via email (instead of using Facebook Messenger)
This section of the comments by participants was related specifically to their overall
experience. Zaption allows for the video to be full screen and to slow down or speed up. Two of
the above comments are related to these two options in Zaption. These comments allow us to
conclude that the navigation bar may have needed prior training before the starting of the
module.
Summary
The instructional module used in this study was intended to provide participants new
skills to benefit them in the future when creating videos or presentations. The studys purpose
was to support our hypothesis that participants will score better on their post-test after watching
13
the instructional module. Based on the pre-test and post-test, analysis of data, and survey
answers it was established that our assumptions were correct. Participants acquired new skills
and demonstrated a change in behavior after viewing the instruction on How to Create a
PowToon. It was also noted based on the usability report, that 10 out of 11 participants agreed
that the lesson was enjoyable.
Furthermore, we observed that, for one participant, the interactive sections were too
numerous. Another participant felt that the screen was too small which may have caused loss of
interest. Taking into consideration all suggestions and recommendations to improve for future
projects, we recognize that learning is a change in behavior. The data of our analysis confirmed a
change in participants, which is what we set out to achieve by creating the instructional module.
The process of finding participants, creating tests, formulating the data and analyzing it was an
educational process. Moreover, we recognized that it is a very critical process as it needs to
provide valid and usable data for accurate results.
14
Appendix A
Invitation to Participate
Hello friends,
We need your help!
My class partner and I need users to help us create an evaluation report for our current class.
Your help is appreciated. This is all done online. You will learn how to use a software to create
an animated video or presentation. You were specifically picked because of your professional
and educational backgrounds and we believe you will enjoy this program and the new
knowledge.
Your part consists of the following steps:
1. Respond to demographics questions and take a pre-test about a software we will
present to you (5-8mins)
2. Watch the instructional video (10:54mins)
3. Take a post-test (same questions as pre-test) (5mins)
4. Respond to a usability report. (5 mins)
This is all simple, quick, and will greatly help us finish our project and hopefully get a good
grade. We will send you the information in steps. Once you complete each, we will send you the
next. We have only till Tuesday to complete this portion (testing).
Please respond via email to confirm your willingness to participate. You can reach us via email
or respond to Facebook message.
Nawara : nnassan@csumb.edu
Claudia: clopez-gallagher@csumb.edu
Thank you for your time.
Nawara & Claudia
15
Appendix B
Pre and Post-test
Please enter name
Please enter your email
Gender
What is PowToon?
A software to create music
A software to create animated presentations
and videos
A software to create PowerPoint presentations
Im not sure
16
17
Appendix C
Invitation to Continue with Part 2 of the Educational Module
Thank you for completing the Pre-Test. Now part 2 consists of 3
things.
1. Please watch this video:
https://www.zaption.com/lessons/5748bef783451ede381e4fa3 [1]
2. Please complete the Post-Test (same questions as the Pre-Test):
https://docs.google.com/a/csumb.edu/forms/d/1ZdrCpIyEJMX0bfzc1kd_F3KTyzU0DUQZEy720ljVBs/edit?usp=sharing
3. Please answer a quick final questionnaire:
https://drive.google.com/open?id=1iBAE3rYumuZ9x5BdxQRrIUnqzfrut2hw---ftqVj6Tw[2]
Thank you so much for your time!
Appendix D
Instructional Module Screenshot
18
Appendix E
Usability Report Questions
Instructions in the presentation are clear
19
20
21
Appendix G
Grading Rubric
Category
Score
20 Pre-Test Questions
20
20 Post-Test Questions
20
Total
40
22
References
Gredler, M. (2009). Learning and Instruction: Theory into Practice. Upper Saddle River, New
Jersey: Pearson Education, Inc.
Kirkpatrick, D. L. & Kirkpatrick, J. D. (1994). Evaluating training programs: The four levels.
San Francisco, CA: Berrett-Koehler.
Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations a systematic approach: a
systematic approach to enhancing learning, performance, and change. New York, NY:
Basic Books.
Stolovitch, H. D. & Keeps, E. J. (2011). Telling aint training (2nd ed.). East Peoria, IL: Versa
Press, Inc.