Professional Documents
Culture Documents
assessment in K-12
classrooms.
Technology can now amplify every aspect of great teaching, which is the key to
raising student achievement. Jimmy Sarakatsannis McKinsey & Company
Begin with technology that makes a difference Mark Stanley, CEO Literatu
January 2016
Page |2
CONTENTS
Executive Summary ..................................................................................................................................................... 3
Introduction ..................................................................................................................................................................... 5
Formative Assessment interactions and data collection ..................................................................................7
Fully adaptive assessment capability in Literatu ............................................................................................7
Insightful reporting aligns to Student learning levels .................................................................................. 12
Structured Assessment - transform and re-engage existing resources...............................................15
Achieving Time efficiences in Marking and Assessment Management .................................................... 18
Short and long form answer marking / grading .............................................................................................19
Live Monitoring of Student Assessment Interactions.................................................................................19
Learning Analytics deliver Visible Learning Metrics .......................................................................................... 21
Paper and LMS based assessments lose Valuable Learning Data ........................................................ 21
Harnessing and visualising large volumes of Learning Data .................................................................... 21
Data stories are everywhere our mission is to expose them .............................................................. 22
Built in analytics ..................................................................................................................................................... 22
Integrate Data Sources for powerful data powered insights .................................................................. 22
Integrate Literatu with existing Systems and Infrastucture. ....................................................................... 24
Instant availability of Literatu both Offline and online modes................................................................ 24
Conclusion.................................................................................................................................................................... 25
Page |3
EXECUTIVE SUMMARY
It often feels as though our schools are designed as production line factories whose goal is to create
a single product (the successful graduate) with the same attributes time after time. Its an old
metaphor but one that continues to be used by technology commentators as an insightful
introduction into a discussion about how technology will improve the current state of education. If
only we could all grasp 21st century learning!.
Maybe it is time to break tradition, right now, and stop typecasting K-12 education as battery hen like
business. Using negative metaphors to start a new discussion doesnt work - just saying. Maybe we
need to look at what technology can do in education and how it can purposefully help teachers extend
skills development and student learning.
The latest OECD report Students, Computers and
Learning found using computers in schools is often
Schools have more data at their fingertips
associated with lower results. Students who use
computers very frequently at school do a lot worse
than ever before, so why does individual
in most learning outcomes. It seems that
learning need to remain such a mystery?
technology needs to do more than move digital
content around networks and talk up impacts that
The solution begins by bringing all of the
are contrary to research findings. Its almost like
information together to form a meaningful
some kind of revenge research is counterpicture of the student.
balancing what we hear from education technology
researchers. Understanding that teaching in K-12 is
best done by teachers might help technology types
to focus on doing more to help teachers improve learning.
Professor Robert Coes research in 2013 proved that whatever technology is doing in education, it is
not making anyone measurably smarter. Three years later, research is still on the same point. Coes
study found that feedback, homework and teachers still make the biggest impact on K-12 student
outcomes (for the least cost), classifying technology as may be worth it. Hatties masterful metaresearch also agreed that teachers and feedback made the biggest difference, not technology.
In their seminal article, Inside the Black Box, Black and William (1998, p.8), concluded Indeed it is
clear that instruction and formative assessment are indivisible. We believe this is where technology
can and should have a more operational focus. Helping teachers save teaching time, target teaching
with live learning data and personalise learning for each student is where technology needs to arrive
and assist in every K-12 classroom. That would be AAA to coin a phrase!
The good news is that in most classrooms, formative assessment practices are already aligned to
instruction. There is however, a twofold tragedy that lurks below. Firstly, many assessment
interactions are designed in a one set fits all context, fuelling the battery hen metaphor and not
helping to extend or remediate students with varying levels of capability and learning need. Secondly,
Page |4
many assessments are still paper based. Photocopiers remain atop of the most used school resource
list, year on year. Not only do teachers continue investing record levels of marking and grading time
in manual assessment processes, but critical learning data and feedback is being lost, daily.
Schools have more data at their fingertips than ever before, so why does individual learning need to
remain such a mystery? The solution begins by bringing all of the information together to form a
meaningful picture of the student. Technology does manage data and information really well.
The Grattan report 2015 highlighted a range of studies showing that at any given year level, there is a
five to six year difference between the most advanced and the least advanced ten per cent of
students. This is a significant challenge for teachers. Technology should be helping teachers better
assess, measure and personalise student learning, period.
Technology now offers different approaches to learning and assessment that are becoming
increasingly valid and more capable of engaging a range of students at their particular level of learning.
We believe that the future of education lies in the delivery of the classroom of one where student
learning and assessment is personalised to meet students at their point of need. In this scenario,
individual learning growth drives measurement, data adapts learning and the best schools and
teachers are those that enable every student to make the greatest progress in learning, regardless of
where they start fromor what device they use.
If we can aspire to a personalised education future, the conversation about how teaching and learning
can be supported with technology becomes exciting and positive.
Page |5
Across both styles of assessment, Literatus core mission is to manage the live formative results data
needed to adapt learning and drive personalised outcomes. Having visibility of detailed learning data
is what teachers have strived to access since the invention of blackboards and chalk.
Literatu is a digital assistant for teachers. Delivery of adaptive assessment via BYOD is instant,
marking and feedback time is immediately reduced, and live assessment monitoring uncovers
learning gaps as teaching happens. Student response and learning measurement data is captured at
every opportunity and made visible via dashboards and extensive analytic reporting. Integration with
Microsoft Power BI extends everyones ability to visualise learning stories that are happening right
now. The main point in all this can do? Teachers can now target teaching and personalise student
outcomes.
Considering it is now 2016, (the other inflammatory phrase designed to start feet tapping as we wait
for action), and that supportive technology is available - hardworking, smart digital assistants, like
Literatu, can support teachers, teaching and learning.
INTRODUCTION
If we believe that education should meet each student's academic needs, why wouldn't we use
assessments that adjust to their individual achievement levels? (G. Gage Kingsbury, March 2014 |
Volume 71 | Number 6 )
The answer to this question is obviously no reason why we wouldnt. The implications of the answer
naturally lead into a discussion about what is involved to deliver adaptive learning and assessment.
Most teachers agree that personalised learning is beneficial to students. Actually being able to deliver
and manage personalised learning is what challenges them. Enabling teacher-driven adaptive
assessment in the context of the classroom of one is what this white paper discusses.
Conventional paper and static digital materials continue to drive the engagement of formative
assessment. Irrespective of the diversity offered by online content sources, checking and reviewing
student assessment, marking and providing feedback - all continue to be predominately manual
processes. Research agrees that feedback and formative assessment are the most effective
interventions a teacher makes so there is no denying the value that teachers add right here.
The time and motion required to manage effective assessment is significant, as is the volume of data
teachers generate every year. Its important to understand the simple maths on how learning data
adds up and just how much learning data is lost across time and in motion.
For example, a teacher with five classes of thirty students, or one class of five subjects, has a base
factor of 150 collection points (30*5=150). When multiplied by ten formative measurement points a
week per student, thats 1500 data points, (10*150=1500). Multiply that by 35 school weeks, that
extends to 52,500 data points. In our research this figure is at the low point. The same 52,500 data
points, if collected and made visible, can help to target teaching, make learning insights actionable,
adapt student learning and most importantly, help students take control of their own learning.
Page |6
Assessments best suited to guide improvements in student learning are the quizzes, short answer
and Bloom-aligned higher order interactions teachers test on a regular basis. If the assessment
process is adaptive to each students skill and level, assessment data is even more accurate and
valuable. Teachers trust the results from assessments they set because of the implicit direct
relationship to classroom instructional goals. Results from every assessment should be immediate
and meaningful to both teachers and students.
Visibility of each students individual learning need is the critical information teachers seek. The data
collected from assessment delivers instant visibility across who is excelling and needing enrichment,
who is performing on target, and who needs help and remediation. Teachers are looking for ways to
help them personalise and manage student achievement and feedback, without learning a new
language and proprietary technology.
The objective of this white paper is to provide an overview of the Literatu platform, how Literatu
helps teachers and integrates into existing school technology eco-systems, in a lingua franca.
Objectives.
1.
2. Achieve time efficiencies and insights into learning measurement. Monitor live
student performance and enhance student feedback.
3. Deliver visible learning analytics, allowing teachers to target teaching and learning to
meet students at their point of need.
Page |7
a pool of questions to draw from with an ability to easily add to the pool
calibration to a common measurement scale
a measurement scale that self-calibrates based on the community of students
a mechanism to select questions on the basis of the student's responses
a process to score the student's responses
a process to feedback progress within time boundaries, or not
feedback that relates scores to the student's instructional needs
Adaptive assessment has been proven to offer key advantages in K12 education. (Kingsbury &
Houser, 1998; Thissen & Mislevy, 2000)
In most studies, adaptive assessment has been found to be as accurate as fixed-form tests
that are twice as long. This enables the assessment to have fewer questions and to take
less time while still providing good information about student achievement.
2. Studies also reveal that adaptive assessment drawing from large item pools can provide
much more information, and more precise information, than fixed-form assessment does
about both students who are struggling and students who are excelling.
3. Assessment allocations calibrate to appropriate learner-levels, continually looking to extend
learning when students are ready for extension, managing them when they are not.
1.
Page |8
4.
Page |9
ASSIGN QUESTIONS OR INCLUDE THEM IN AN ADAPTIVE POOL OF QUESTIONS
Any question, from any question library, can be assigned in a structured activity or as part of an
assessment pool of questions. There is a range of options to change the way questions are
presented, scored and managed.
P a g e | 10
Questions can be selected at random or by choice, previewed and re-ordered, very quickly.
Questions from any library can be included into any assessment. When schools have access to
multiple publisher content libraries, they can choose any number of parameters to create an
adaptive pool. Teachers can also simply select a level of the curriculum and let the adaptive engine
do the rest.
P a g e | 11
ADAPTIVE ASSESSMENT QUESTIONS AND ADAPTION
Every question is linked to a curriculum level, a skill and possible sub skill, a year of study and a level
of difficulty. The level of difficulty is aligned to Blooms higher order cognitive skills with three levels
of difficulty for each cognitive level. The level of difficulty is used as a parameter in the adaptive
algorithm when assigning the next
most appropriate question to
students.
If the level of difficulty set for the
question is wrongly assigned,
Literatus machine learning
algorithm will auto correct based
on the pattern of student answers. This feature maintains the perpetual levelling of question
difficulty, ensuring the adaptive engine has access to the most current statistics from the cohort.
This feature ensures questions stay relevant and accurately applied.
P a g e | 12
needing to be established at the content / question library level, meaning anyone can start building
questions and making them available in Adaptive Learning deployments, in minutes. Publishers can
bulk load questions in a number of ways.
P a g e | 13
A Longitudinal trajectory of learning is monitored across many levels, certainly in the context of the
class, across a range of assessment activities and dates.
Importantly, its not all about right and wrong answers. Teachers need to know what skills are
lacking and possibly blocking learning from progressing. Literatu also tracks skills linked to
assessments across time lines, showing where cohorts need more support. Box plots show upper,
lower and quartile levels across the cohort.
P a g e | 14
All data from Literatu is available for further analysis in Power BI.
In the words of W. Edwards Deming, If you dont have the data you need to make a
decision, you are working with a personal opinion
P a g e | 15
STRUCTURED ASSESSMENT - TRANSFORM AND RE-ENGAGE EXISTING
RESOURCES.
This is where Literatu initially started working with teachers, helping them to share and re-purpose
existing trusted resources as the base of improving assessment intelligence, live monitoring and data
insights. This capability is fully supported in the platform and is referred to as Literatu Classic the
structured assessment model. The proposition is still unique. From any written paper or PDF digital
source, Literatu lets teachers quickly transform assessment materials into interactive, online
activities.
Three steps quickly transform static, digital assessment content to live online student interactions.
1. Load any digital file in PDF format.
2. Overlay interaction points, into which students record their answers. Establish marking
criteria, answers, guidance and feedback to any suitable level.
3. Send the assessment for completion across any BYOD device. Workflow manages the
complete assign, hand in, marking, feedback, and hand back process. Reporting and
analytics support the process.
Three immediate benefits realised.
1. Trusted assessment resources stay aligned to existing pedagogy. Teachers have continuity
of materials and context in which they are used. No change.
2. Marking and feedback processes are streamlined with significant time savings delivered.
Students are engaged in any BYOD environment, online or offline.
3. The significant volume of student response data captured is visible through online analytics.
Combined with live monitoring, teachers have full oversight across student learning gaps
and opportunities to help target their teaching.
P a g e | 16
LOADING ASSESSMENT RESOURCES INTO LITERATU ONLINE LIBRARIES
Each school is an online community in which teachers share and manage their assessment content
libraries. Any PDF, Microsoft Word, PowerPoint, Excel or native Google Document can be loaded
into the Literatu Activity Library, made interactive and shared amongst colleagues.
P a g e | 17
The Literatu web portal operates in 5 pre-loaded languages. English, Malay Bahasa, Indonesian
Bahasa, Japanese and Mandarin. Additional languages are quickly added by uploading a translation
file. Each user can nominate the language in which Literatu will display. Chrome, Edge, IE10+ and
Firefox are all supported.
P a g e | 18
Teacher defined tolerances (Red / Green / Amber) set levels, show progress scores, completion
statistics, and the marking / grading required. Each activity is further analysed down to question level
and then to individual student performance. Bubble size and colour highlight the range and size of the
answer spread. Our trademark reporting is now our famous motto, the bigger the bubble, the bigger
the trouble.
P a g e | 19
SHORT AND LONG FORM ANSWER MARKING / GRADING
Literatu supports the marking of both short and long form answers, reducing teacher review and
feedback time. Teacher marking is done via web or by the mobile teacher application. On tablet
devices, teachers have access to stylus enabled marking, able to write directly over the student
submission.
Marking can be done by question / by student, by each student individually, or both ways. Each
response can be checked for plagiarism and writing style discrepancies. Recommended marks are
suggested based on the usage of pre-set answer keywords.
P a g e | 20
Monitor student work in detail, as they are working. Leave live comments on their work, as they
work. For any questions that need a marking adjustment, make it right away.
P a g e | 21
P a g e | 22
As an example, imagine being able to look across cohort results for all writing based subjects and
compare those results against English or look for skill deficiencies by cohort prior to embarking on
new streams of curriculum that require pre-requisite knowledge. Once questions can be asked,
questions never stop and nor should they. Data informed teachers are better teachers.
DATA STORIES ARE EVERYWHERE OUR MISSION IS TO EXPOSE THEM
Volumes of data are of no use unless there is a way to see a story, find a trend or have that ah-ha
moment where something finally makes sense.
Literatu builds large data sets and presents them to a varied data audience in two ways.
BUILT IN ANALYTICS
The best type of data for teachers is data that is about them, their class, context, curriculum and
students. At any one point in time, data is both current and complete. Instant access to all of the data
within the teachers context is optimal.
Literatu builds instant operational data views for teachers across subjects, classes, curriculum and
students. These charts are live, surfacing data as results are recorded. Instant access to learning data
supports teacher agility and helps students take responsibility over their own learning. There is a range
of teacher, faculty and school based analytic reports that are available online as student interactions
move from in progress to complete.
P a g e | 23
common data attribute collected across all interactions, is student name. This is at least
encouraging.
From summative assessment data sources, to adaptive formative assessment results, student
learning data is usually abundant in supply. The true value of this data is in the aggregation and
analysis of this data.
Literatu opens all of its data collection capabilities to the power of Microsoft Power BI. Via an active
API data link, schools can quickly move assessment data from Literatu into Power BI data models,
ready for analysis and visualisation.
Power BI extends the power of data aggregation to every organisation, truly fixing what used to be
called data aggravation. Consolidating data sources to visualise and discover data stories offers
positive and rewarding outcomes. Being a very agile component of Office 365, Power BI is web
delivered and built for anyone with a base understanding of Excel, to enjoy data discovery.
Power BI can report across large swathes of data, driven by English syntax questions and queries.
Asking the question, Which teacher is getting the best results for Chemistry? is now as simple as
asking Power BI the question.
Power BI also manages the distribution and accessibility of data dash boards, making reporting
available on mobile phones and tablet devices. There really is no excuse for teachers not to have up
to date information at their fingertips.
P a g e | 24
P a g e | 25
CONCLUSION
If teachers can be better supported by technology, teachers can spend more time teaching.
It sounds simple but thats what technology needs to make happen. To help teachers spend more
time teaching, we looked at where technology can make significant contributions. The biggest
contribution Literatu technology makes to teachers is by providing adaptive assessment capabilities
to deliver data-informed insights into where teachers should target their teaching to meet students
at their point of learning need, every day.
Data informed insights need to be powered by technology because teachers dont need more work
to do. Technology needs to be the digital assistant for teachers, managing the critical relationship
between instruction and assessment. Technology needs to measure student learning, adapt to
student learning need, collect evidence of achievement and build data insights.
Literatu does this well because we get it Literatu is technology that better supports teachers.
LITERATU
GO TO : www.literatu.com