Field Test Briefings

September 2014

Who is in the Room?

MAEIA Project Team Member(s)
Field Test Teachers
Name
Role
District
In a Word.Describe your interest/motivation in
trying out MAEIAs assessments in Dance, Music,
Theatre or Visual Arts

Outcomes for the Day

Understand what MAEIA is and what the model

assessments are
Understand what items to select and try out
Know how to order the items
Understand how to administer the assessments
Understand what materials to return and how to
return them
Know that there is a request and expectation for
feedback on the items and the process
Know who to contact if there are questions
3

Agenda for the Day

Introductions
Overview of the MAEIA project & timeline
Explanation of the Catalog Assessments
Selection of Items
Lunch Break
Ordering
Activities Before, During, and After Administration
Return of Materials
How to Use the Assessments
Questions
Who to Contact with Questions
SCECHs Sign Out
4

Overview of MAEIA
Prezi Presentation on the MAEIA Project

MAEIA Three-Year Plan

Year 1 2012-2013
Develop and Publish Arts Education Instructional Blueprint and Supporting
Research
Develop Program Review Tool
Create the Assessment Specifications Document
Year 2 2013-2014
Field Test the Program Review Tool
High School: Item Writing, Content Review, and Booklet Production
Professional Development: Year 1 Tools
Start K-8 Item Writing
Year 3 2014-2015
HS: Field Test
K-8: Item Writing, Content Review, and Booklet Production
Professional Development: Year 1 and 2 Tools
Revise and Refine HS Items (scoring tools, attach exemplars)
Year 4 2015-16
K-8 Field Test
Professional Development Year 1, 2 and 3 Tools/Resources
8

Intro to Assessment
Four disciplinesDance, Music, Theatre, and
Visual Arts
Assessments could be used in several ways:
Informing current instruction
Future instructional program improvement
Improved student learning and achievement

As a portion of educator evaluation

The Integral Role of

Standards in the Assessments
Based on MAEIA Performance Standards
Assess important content, knowledge and skills in the
Michigan Arts Education Standards and Benchmarks
2011
Organized as Create, Perform and Respond

Performance Standards
Items also are aligned to the VPAA Credit
Guidelines

10

Layers of Standards
MAEIA Performance Standards
Michigan Content Standards and Benchmarks
Michigan Grade Level Content Standards

11

Example of Layers of
Standards

12

Performance Standards are based on the 2011 edition of the

Michigan Content Standards and Benchmarks, which can be
found on the MDE Website or the Mi-Arts Wiki

13

Chapter 4 Nature of the Assessment Items

Types of Assessment Items

Performance Tasks

Performance assessments carried

out by individual students or small
groups of students over time (days,
weeks, months)
Performance Events

Performance assessments that are

administered on-demand, without
any or just a brief amount of
rehearsal time.
14

Performance Tasks
Performance assessments carried out by individual

students or small groups of students over time

(days/weeks/months)
Tasks are carried out in or out of class, and are very
much related to instruction (e.g., class assignments)
They measure essential outcomes in the content
standards not easily measured in other ways
Performances are judged using one or more scoring
rubrics

15

Performance Events
Performance assessments that are administered on-

demand, with limited or no practice or rehearsal

time (i.e., no more than a few minutes)
May be individual or small group assessments
Test administrator presents items to one student or a
small group of students, who respond in real time
Performances are also judged using one or more
scoring rubrics

16

Chapter 4 Nature of the Assessment Items

Types of Assessment Items contd

Constructed Response

Items in which students

write a response to a
prompt.
Selected Response

Items related to other items

in which the student is
given a prompt (a
question or a statement)
and answer choices
17

Constructed-Response
Items
Items in which students write a response to a

prompt
Usually can be administered to groups of students
together
Some type of stimulus (e.g., music selection, video,
or picture) could be used
Task may involve writing, sketching, constructing a
table, as well as a written response
Performances are judged using one or more scoring
rubrics
18

Selected-Response Items
Items in which the student is given a prompt (a

question or a statement) and answer choices

Student has to select either the correct answer or the
most correct answer
Multiple-choice questions are the most popular form
of these items
In MAEIA, these will be used to tap the content
knowledge or other procedural knowledge needed
to respond to PT/PE/CR items
19

Why do we field test

assessments?
We follow Best Practice by following the professional
measurement standards that govern the development &
use of assessments:
Are they written so that teachers and students
understand them?
Do they yield assessable information?
Do the items assess what they are intended to assess?
Have the scoring guides been reviewed and revised (can
they be strengthened)?
Any assessment with stakes for teachers and/or students should
be piloted or field tested before it is used to make important
decisions.
(AERA/APA/NCME Standards for Educational and Psychological Testing)
20

Introduction to the Catalogue

of MAEIA Assessment Items
DisciplineDance, Music, Theatre, or Visual Arts
Item Type and NumberThe code for each task (T) or event (E).

Note: all High School assessments are the 400 series. This will
distinguish them from grade K-2 assessments (100s), grade 3-5
assessments (200s), and grade 6-8 assessments (300s).
Short Item DescriptorA brief phrase or title of the assessment,
intended to convey the essence of the item.
Grade RangeThis is always set at 4, i.e., high school.
Levels 1, 2 and 3This designates the level or levels for which the
item was written.
Performance StandardPerformance standards are disciplinespecific and condense the Michigan Merit Curriculums five content
standards into the three overarching performance standards of
Create, Perform, and Respond. These performance standards were
developed to serve as the basis for MAEIA model assessments that
measure student proficiency in each arts discipline at each grade
span - K-2, 3-5, 6-8, and 9-12.
21

Introduction to the Catalogue

of MAEIA Assessment Items
Content Standard and BenchmarkThese are the Michigan State

Board of Education-approved standards for arts education, taken

from Michigan Arts Education Content Standards and Benchmarks for
Dance, Music, Theatre and the Visual Arts (2011).
VPAA GuidelineThese are taken the Michigan State Board of
Education-approved Michigan Merit Curriculum Credit Guidelines for
the Visual, Performing, and Applied Arts (2006).
Requires RecordingThis is a signal to a teacher using this item
about whether students responses will need to be electronically
recorded.
Total TimeThis is an indication of the number of 50-minute class
periods that the assessment requires. This number may not be
precise for two reasons:
Your class period length may differ
The time it takes to record and download the performances of all

students in larger classes may require an extra class period (or two)
22

Assessment Levels
Because students participate in the arts for different reasons and
durations, assessment tasks and events were developed for three
different levels of HS arts students:
Level 1Students who fulfill their one credit visual, performing, or applied arts
(VPAA) high school graduation requirement only, or who are in their first year of a
multi-year VPAA program.
Level 2Students who have already completed their first year in an arts discipline
course and are now in their second year of instruction in the same arts discipline.
Level 3Students who have already completed their first and second years in an arts
disciplines courses and are now in their third year or fourth year of instruction in the
same arts discipline.

If a student takes a year of instruction in one discipline (e.g., music)

and another year of instruction in another discipline (e.g., theatre), they
would participate in Level 1 assessments in each discipline. Teachers
have the flexibility to adjust the assessments to match the instructional
levels of the students being assessed.

23

Steps in Selecting Items

A Few Words about the MAEIA Arts Education Assessments
There are about 30+ Events and Tasks in each discipline. (Dance, Music, Theatre, Visual Arts)
These measure the three state and national arts education standards of Perform, Create, and
Respond; more specific skills are measured by each Event and Task.
Events usually take a class period to complete; Tasks take multiple class periods to complete.
We are expecting teachers to select a small number of Events and Tasks to use throughout the
school year (OctoberApril).
We are suggesting that items be used within three instructional cycles, to help you pace your use
of the Events and Tasks throughout the school year:

October 15 December 15, 2014 (fall)

January 7 February 27, 2015 (winter)
March 2 April 17, 2015 (spring)

We are suggesting 1-2 Tasks and 1-2 Events per Instructional Cycle. This is a recommendation, not
a requirement; field testers may select fewer (or more) items to try-out.
You may modify the assessments to better fit your instruction (e.g., change the genre of the work
of art used in the assessment). You will tell us about any modifications you made.
After using and scoring the assessments yourself, you will return student responses to us to aid in
refining the items and enhancing the scoring rubrics.
Your feedback on the uses and usefulness of the assessments is very important to us. We look
forward to your feedback as you finish using the assessments during each instructional period and
at the conclusion of the field test.

24

Steps in Selecting Items

Step 1

Look at the Catalog for your discipline and the Short

Item Descriptor of the Tasks and Events. Use one
color highlighter to check or highlight the Tasks or
Events that interest or intrigue you, i.e., that you
would like to know more about.

25

Steps in Selecting Items

Step 2
Now, click on the hyperlinks for the Short Item Descriptors in the
Catalogue for the items you selected in Step 1. Look at the Michigan
Student Learning Standards Assessed, Level, Overview and Outline,
and the Suggested Total Time for the items checked. You should select
the Tasks and Events that will logically fit into your existing
curriculum and instructional plans. Use the other color highlighter to
note
these.
Item
Type
T = Task E = Event
Grade-Range

4 = High School (only ones available this year)

Level

1 = Year One Students

2 = Year Two Students
3 = Year Three and Four Students

Total Time

Number of 50-Minute Class Period

26

Steps in Selecting Items

Note the Michigan Student Learning Standards Assessed

that the item assesses to determine if you think this

assessment item meets your needs with respect to
expectations for the course in which you would use the
item (remember you may modify the item e.g. genre
if necessary).
Note the Level of the item to determine if you think the
item will reflect the level(s) of students you will have in
the course in which you will use the assessment. Level
refers to the number of years that students have studied
in the discipline in high school, including the current
school year.
Note the suggested Total Time to Assess (found in the
Catalog and in the Teacher Booklet). Within which
instructional cycle(s) and when during the instructional
cycle might this item fit?

27

Steps in Selecting Items

Step 3
Place possible selected items to field test into the Instructional
Cycles to visually check to see if you have a possible suite of
items that will result in a reasonable administration plan for
the courses in which you plan to administer the items.
The table shows the information you will need to identify to
place your order for items. This includes:
Instructional Cycle
Item Type and Number
Total Number of Students to be Assessed

With the addition of Coursenoting which course(s) you

intend to use each item inyou now have a field test
administration plan (for your records).
When you are satisfied with your administration plan, Proceed
28
to Ordering Items!

Steps in Selecting Items

My#Field#Test#Administration#Plan#for#MAEIA#High#School#Arts#Assessments#
!
Teacher!Name:!!______________________________________!
School!Name:!!!!!______________________________________!
District!Name:!!!______________________________________!

!
Item#ID#
Number#and#
Description#

Course(s)#in#
which#I#plan#to#
use#this#item:#

Instructional#
Instructional#
Instructional# Total#number#
Cycle#1#(Fall)# Cycle#2#(Winter)#
Cycle#3#
of#students#
October!15!!
January!5!@
@
!!
(Spring)#
(across!all!
December!15,!
February!27,!
March!1!!
cycles)#
2014#
2015#
April!17,!2015#
!
!
!
!

!
!

!
!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

!
!

29

Steps in Selecting Items

Additional Considerations
There is a separate Teacher Booklet for each assessment. Each item is
self-contained
You will be sent a sufficient number of Student Booklets (plus a
small overage) and couple of copies of the Teacher Booklet for each
assessment item you select
You will receive an Assessment Administration Manual (AAM) with
overall field test directions in it. The AAM discusses general
assessment topics, including actions you should take before, during,
and after assessment.
We encourage you to select a mix of Create, Perform and Respond
Items. Combined, these strands are the creative process (in Visual
Arts, we think of Perform as Present.)
Any additional materials needed to administer an item is noted.
Recording of student responses is also noted.
30

Assessments Not Yet Available

These Theatre and Visual Arts tasks and events are not yet
available, pending obtaining permissions:
Theatre
T.T417
T.E409
T.E416

Visual Arts
V.T401
V.T409
V.E413
V.E414
Feel free to order any of these. Once permissions have been
obtained, these assessments will be sent to the field test teachers
who selected them.
31

Individual Review and

Selection
Use this time to go through the steps 1,2, and 3 to

review the available Tasks and Events from the

Catalog in your discipline
By lunch, we hope you have tentatively selected the
Tasks and Events to investigate further (Step 1),
examined the information for each of these items of
interest (Step 2), and selected the Tasks and Events to
use and slotted these into each of the Instructional
Cycles (Step 3)
After lunch, well show you how to place your order
and then discuss more about the overall field test
32
activities

Lunch 12:00 - 12:30

33

Ordering Your Items

First, you will enter contact and shipping information.
This is very important!

This screen continues in the actual order form.

34

Ordering Your Items

Next, you will select the discipline in which you will be
field-testing assessment items.

35

Ordering Your Items

Select instructional cycle(s).

Select the tasks

and events.
Click on
the little
arrow that
looks like a
v to
select
number of
students.

36

Ordering Your Items

This is the last screen you will see before you submit
your order. Please be sure your order is correct before
you click SUBMIT. Thank you!

37

Assessment Administration
Manual
TABLE OF CONTENTS
Section
Letter from the Michigan Assessment Consortium
Table of Contents
Chapter 1Overview of the MAEIA Arts Education Resources and Their Uses
1.1Overview of the MAEIA Project
1.2Benefits and Uses of the MAEIA Arts Education Resources

Chapter 2Purposes of the MAEIA Arts Education Assessments

2.1Purposes and Intended Uses for the Arts Education Assessments
2.2Cautions in the Use of the MAEIA Arts Education Assessments
38

Assessment Administration
Manual
Chapter 3Overview of the Arts Education Assessment Design
3.1Discipline Areas to be Assessed
3.2Nature of the Assessment Items
3.3.1Performance Tasks
3.3.2Performance Events
3.3.3Constructed-Response Items
3.3.4Selected-Response Items
3.3Use of Graphics, Audio and Video in the Assessments
3.4Accessibility and Accommodations Universal Design (UD) Evidence-Centered
Design (ECD)
3.5-Assessment Time
3.6Assessment Levels

39

Assessment Administration
Manual
Chapter 4Selection of the Assessment to Administer
4.1Catalog of Available Assessments
4.2Fitting the Assessments Into Your Instructional Program
4.3Selecting When to Use the Assessments
4.4Adding Additional Assessments
4.5Deleting Planned Assessments
4.6Questions or Concerns
40

Assessment Administration
Manual
Chapter 5Assessment Administration Procedures
5.1Overview of the Assessment Administration Process
5.2Pre-Assessment Activities
5.2.1Determine Students to be Assessed
5.2.2Inventory the Assessment Materials Ordered
5.2.3In Case of Materials Shortages
5.2.4Briefing Parents and Students
5.2.5Preparing for the Assessments
5.2.6Modifying the Assessments
5.2.7Questions and Concerns
41

Assessment Administration
Manual
Chapter 5Assessment Administration Procedures
5.3Activities During the Assessments

5.3.1Observation of Students During Assessment

5.3.2Recording Students Responses
5.3.3Assistance to Students During the Assessment
5.3.4When Issues Arise
5.3.5Feedback to the MAC from Students and the Teacher

5.4Post-Assessment Activities
5.2.1Uploading Electronic Responses to the Flash Drives
5.4.2Scoring the Assessments
5.4.3Providing Feedback on the MAEIA Assessments
5.4.4Packaging Loose Ancillary Student Responses
5.4.5Packaging the Used and Unused Materials for Return Shipment
5.4.6Completing the Materials Return Cover Sheet
5.4.7Returning the Assessment Materials
42

Assessment Administration
Manual
Attachments
AAssessment Catalog
BItem Review and Comment Form
CMaterials Return Cover Sheet
DInstructions for Transferring Audio/Video from
Device to Flash Drive via Computer
EMaterials Return Label

43

Individual Review Chapters 4

and 5
Read Chapters 4 and 5 of the Assessment

Administration Manual
Note any procedural questions to raise
Raise these questions with the Briefing facilitators

44

Before Administering the

Assessment
Inventory the assessment materials you were sent. Do

you have enough Student Booklets? Order extras from

DRC is needed.
Read through each assessment one or more times
Practice administering the assessment with a colleague or
to yourself
Assemble other materials needed (if any)
Decide whether the assessment can be used as is or needs
to be modified. You may modify them if necessary.
Determine how students responses will be recorded and
downloaded (if necessary)
45
Determine when the assessment will be used

During the Assessment

Observe students during the assessment and note

any issues, confusion, or questions on their part

Video or audio record students responses to the
assessment where indicated
Determine how these recorded responses can be
saved during the assessment and submitted at the
conclusion of the assessment
Collect the assessment materials between class
periods and at the conclusion of the assessment
Organize the assessment materials for return to DRC
46

After Administrating the

Assessment
Score students responses using the Teacher Scoring

Rubric(s) provided; note any issues with the rubrics

Upload recorded student responses to the flash
drive(s) provided
These are secure assessment materials
Put only responses to one assessment item on a flash

drive; one flash drive has been sent to you for each 25
students indicated on your original order
Put the flash drive(s) in the envelope provided; fill out
the label(s) provided and seal the envelope
47

After Administrating the

Assessment

48

After Administrating the

Assessment
Place the loose ancillary student responses in the

envelope provided and complete the Ancillary

Student Materials Return Envelope label.
Ancillary materials include the following:
Assessment Questions pages that students are

instructed to tear off and give to you at the start of

many of the assessments
Drawings/paintings done by students
Photographs of student work in lieu of return of large,
bulky responses and/or 3-D objects (e.g., cans or kites)
Any other student responses not done in their Student
Booklets
49

After Administrating the

Assessment

50

After Administrating the

Assessment
Provide feedback on the assessment using the Item

Review and Comment Form sent with the shipment of

assessment materials
Mark up a copy of the Teacher Booklet and/or
Student Booklet with your comments
Place the the Item Review and Comment Form and the
marked up Teacher Booklet and/or Student Booklet
with your comments in the envelope provided. Fill
out the label to indicate the assessment
Include a copy of any revised Teacher Booklet and/or

Student Booklet that you have created in the same

envelope
51

After Administrating the

Assessment
Item Review and Comment Form

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Identification!Information!
District!Name!
!
School!Name!
!
Teacher!Name!
!
Teacher!Identification!Number!
!
Item Type
& Number

Comments on
Assessment
Administration

Comments on Recording
of Student Responses

Comments on Scoring

Other Comments

52

After Administrating the

Assessment

53

After Administrating the

Assessment
Packaging the materials for return to DRC:
Use sturdy clean boxes to ship the materials via UPS (e.g.,
the boxes the assessment materials were sent to you in)
Send each assessment after you have given it and scored
students responses
Use the pre-paid labels to ship the materials to DRC at no
cost to you or your school
If the assessment return requires more than one box, label
each 1 of X, 2 of X, etc., with X = total number of boxes
Fill out a Materials Return Cover Sheet and place on top of

the first box

54

After Administrating the

Assessment
Materials Return Cover Sheet

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Identification!Information!
District!Name!
!
School!Name!
!
Shipping!Address!
!
!
!
Teacher!Name!
!
Teacher!Identification!Number!
!
!

Item!Type! Administration!
&!Number!
Status!
(1)!

!
!
!
!
!

(2)!

!
!
!
!
!

No.!Booklets!
Sent!

Materials!Being!Returned!
No.!Booklets!
No.!Booklets!
Used!
Unused!

(3)!

(4)!

!
!
!
!
!

!
!

(5)!

!
!
!

!
!
!
!
!

No.!New!or!
Revised!Booklets*!

No.!Ancillary!
Student!
Responses!

!No.!Flash!
Drives!

(6)!

(7)!

(8)!

!
!
!
!
!

!
!
!
!
!

!
!
!
!
!

!
Administration!Status!(Column!2)!
For!each!folder!ordered!and!sent!to!the!teacher,!please!indicate!in!column!2!how!you!chose!to!administer!it:!
!
1!=!!Administered!the!assessment!as!given;!no!changes!or!modifications!were!made!by!the!teacher!
2!=!!Administered!the!assessment!as!given,!but!wrote!recommendations!for!modification/improvement!in!a!copy!of!the!Teacher!Booklet!and/or!
Student!Booklet!(and!returned!these!in!the!envelope!provided).!
3!=!!Changed!the!assessment!significantly!and!prepared!a!new!Teacher!Booklet!and!Student!Booklets!in!order!to!administer!the!assessment.!A!copy!of!
the!revised!Teacher!Booklet!should!be!returned!in!the!envelope!provided,!and!the!revised!Student!Booklets!used!by!students!have!been!returned!
(along!with!the!original.!unused!Booklets).!Indicate!the!number!of!new!Booklets*!returned!in!column!6.!
!

!
!

!
!

Completion!of!Columns!38!
Fill!in!the!number!of!materials!that!were!received!(column!3),!the!number!used!by!student!and!being!returned!(column!4),!the!number!not!used!by!
students!(column!5),!as!well!as!the!number!of!teacherPadapted!Student!Booklets!being!returned!(in!column!6).!In!column!7,!indicate!the!number!of!
separate!student!responses!being!returned!(e.g.,!photographs!of!works!of!art,!separate!sketches!or!drawings,!etc.),!while!in!column!8,!the!number!of!
student!response!flash!drives!being!returned;!place!in!speciallyPprovided!enveloped.!
On!the!reverse!side,!indicate!modified!by!the!teacher!(Administration!Status!#3!in!column!2),!please!describe!briefly!the!types!of!modifications,!if!any,!
you!made!to!the!items.!
See!Reverse!Side!for!More!Directions!

55

After Administrating the

Assessment
Modification!of!Folders!
Please!describe!briefly!the!types!of!modifications,!if!any,!you!made!to!the!items.!
!

Item!Type!&!
Number!
!
!
!
!
!
!
!
!
!
!
!
!

!
Summary!of!Changes!Made!or!Recommendations!for!Changes!You!Feel!Should!be!Made!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!

56

After Administrating the

Assessment

57

How to Use the Results for

Educator Effectiveness
If you feel that the assessment represents a good

assessment of students (that is, that students

understood what to do and were able to respond to
the assessment), then the assessment results may
represent their current achievement
Teachers can select students responses to represent
their work with the class (to show what the class was
able to do) as well as individual students (those
whose performance was exceptional or students who
struggled but were still able to complete the
assessment)
58

How to Use the Results for

Educator Effectiveness
Teachers can also select examples of student work

from each instructional period to illustrate changes

in students work over the school year
Students whose performance was marginal in the fall

who are now doing exceptional work

Students who struggled to perform at all who are now
doing acceptable (or better) work
Student who still need help and how you are working
with them

The teacher and the school administrator can use this

work as one piece of evidence of student learning and

achievement
59

How to Use the Results for

Educator Effectiveness
Other sources of achievement data (e.g., students

responses to other MAEIA assessments as well as to

other measures or indicators of achievement) should
be used as well
Prepare corresponding narratives regarding teacher
practices to accompany student assessment results
These achievement data along with appropriate
observational data should be used in the overall
evaluation of an educator
The goal of such educator evaluation should be
primarily improvement of educator practice
60

Cautions about Using the

Results
Remember, this is the first time these assessments have

been given to students

The assessments may have issues that prevent students
from showing you (and us) what they have learned and
can do (the purpose of field testing is to uncover these
issues and to correct them)
The assessment items as written may not fit well with the
instruction planned and implemented by the educator
Even if the assessment item worked well and fits with the
educators instruction, no single data element should be
used to make important decisions about educators or
students

61

What Questions Do You

Have?

62

Contact Us for Help:

MAEIA Field Test Resource Page
http://www.michiganassessmentconsortium.org/maeia
-high-school-assessment-field-test-resources
Jason ODonnell
517.327.9224517.327.0779 (fax)
jodonnell@michiganassessmentconsortium.org

Jason will forward each message to the most appropriate

person to answer your question(s).

63

Closing
Thank you for attending the Briefing, and for

volunteering to participate in the field test!

SCECHs (MAEIA is paying the filing fee)
Evaluation of Todays Briefing from Kent ISD

64