schools of engineering

Creating a Rubric

Subramaniam D. Rajan
Professor of Civil, Environmental & Sustainable Engineering
Professor of Mechanical & Aerospace Engineering
 What is a Rubric?

A scoring rubric is an attempt to communicate expectations of

quality around a task. In many cases, scoring rubrics are used
to delineate consistent criteria for grading. Because the
criteria are public, a scoring rubric allows teachers and
students alike to evaluate criteria, which can be complex and
subjective. A scoring rubric can also provide a basis for self-
evaluation, reflection, and peer review. It is aimed at accurate
and fair assessment, fostering understanding, and indicating a
way to proceed with subsequent learning/teaching. This
integration of performance and feedback is called ongoing
assessment or formative assessment.

http://en.wikipedia.org/wiki/Rubric_%28academic%29
FULTON
schools of engineering
sustainable engineering and the built environment
 Why Rubrics are Important?

A well-planned and well-designed rubric can play a very

important role by:
 Providing informative feedback
 Promoting transparency
 Improving student performance by providing clear
guidelines for assessment
 Making grading more consistent and time-efficient
 Guiding students in peer and group assessment

FULTON
schools of engineering
sustainable engineering and the built environment
 Types of Rubrics

 Checklists
 Basic Rating Scales
 Holistic Rating Scales
 Analytic Rating Scales
Reference:
http://www.gallaudet.edu/office_of_academic_quality/assessment_of_student_learning_outc
omes/instructions_and_examples/developing_a_scoring_criteria_%28rubrics%29.html

FULTON
schools of engineering
sustainable engineering and the built environment
 Checklists

 Checklists: The least complex form of scoring system, are

simple lists indicating the presence, NOT the quality, of the
elements. Therefore, checklists are NOT frequently used in
higher education for program-level assessment. But faculty
may find them useful for scoring and giving feedback on
minor student assignments or practice/drafts of
assignments.

FULTON
schools of engineering
sustainable engineering and the built environment
 Checklists

FULTON
schools of engineering
sustainable engineering and the built environment
 Basic Rating Scales

 Basic Rating Scales: Checklists of criteria that evaluate the

quality of elements and include a scoring system. The main
drawback with rating scales is that the meaning of the
numeric ratings can be vague. Without descriptors for the
ratings, the raters must make a judgment based on their
perception of the meanings of the terms. For the same
presentation, one rater might think a student rated “good”
and another rater might feel the same student was
"marginal."

FULTON
schools of engineering
sustainable engineering and the built environment
 Basic Rating Scales

FULTON
schools of engineering
sustainable engineering and the built environment
 Holistic Rating Scales

 Holistic Rating Scales: Uses a short narrative of

characteristics to award a single scored based on an
overall impression of a student's performance on a task. A
drawback to using holistic rating scales is that they do not
provide specific areas of strengths and weaknesses and
therefore are less useful to help you focus your
improvement efforts. Use a holistic rating scale when the
projects to be assessed will vary greatly (e.g., independent
study projects submitted in a capstone course) or when
the number of assignments to be assessed is significant
(e.g., reviewing all the essays from applicants to
determine who will need developmental courses).

FULTON
schools of engineering
sustainable engineering and the built environment
 Holistic Rating Scales

FULTON
schools of engineering
sustainable engineering and the built environment
 Analytic Rating Scale

 Analytic Rating Scale: Rubrics that include explicit

performance expectations for each possible rating, for each
criterion. Analytic rating scales are especially appropriate for
complex learning tasks with multiple criteria. Evaluate
carefully whether this the most appropriate tool for your
assessment needs. They can provide more detailed
feedback on student performance; more consistent scoring
among raters but the disadvantage is that they can be time-
consuming to develop and apply. Results can be aggregated
to provide detailed information on strengths and weaknesses
of a program.

FULTON
schools of engineering
sustainable engineering and the built environment
 Analytic Rating Scale

FULTON
schools of engineering
sustainable engineering and the built environment
 How to Design Rubrics

 Step 1: Look carefully at the problem statement for the item

that is being assessed.
 Step 2: Identify the skills, knowledge and understanding
that you want the students to demonstrate. One can
categorize these into distinct sets or groups.
 Step 3: Assign a number of performance levels and write
the performance descriptors for each of those levels.

FULTON
schools of engineering
sustainable engineering and the built environment
 Example: Step 1

Example Problem Statement: Student teams have to make

an oral presentation on a design project. The presentation
must focus on a number of technical points with emphasis on
(a) why and how the team built the preliminary design
models, (b) why and what they used as a preliminary design,
(c) how and why they iterated towards the final design, and
(d) lessons learnt (extremely important). The presentation
should focus on unique aspects of the design process,
dissection of the final design and thought-provoking
discussions.

FULTON
schools of engineering
sustainable engineering and the built environment
 Example: Step 2

 How many categories?

– Divide skills and knowledge into a few distinct categories
 For the example under consideration
– Technical Contents: Design problems typically involve one or more
central goals that are maximized or minimized, performance
requirements that must be met, experiments or computer models
that are used to evaluate the design.
– Coherence: Designs evolve iteratively and systematically from an
initial design to the final design.
– Presentation Skills: The design process needs to be described in a
sufficiently high level language for the audience to understand
what was done.

FULTON
schools of engineering
sustainable engineering and the built environment
 Example: Step 3

 How many levels (Analytic Rating Scale)?

– Rating from 5 (Exemplary) to 3 (Developing) to 1 (Beginning)
– Could also define levels 2 (Fair) and 4 (Strong)
 For the example under consideration
– Exemplary: This is the highest level indicating that the student has
met and exceeded all expectations.
– Developing: Indicates that the student is on the way to meeting the
expectations.
– Beginning: This is the lowest level indicating that the student is
having difficulty understanding the material and requires help in
meeting an acceptable performance level.

FULTON
schools of engineering
sustainable engineering and the built environment
 Example: Oral Presentation Grading
Category Rating 5
(Exemplary)
The presentation was original,
Technical clear, and complete. The
Contents speakers exhibited clearly that
they understood the design
problem and that their design
met all the design
requirements.
The design details were clearly
Coherence presented. The audience had
no difficulty in following the
design evolution and
understood the final design
details. The slides were clear
and error-free.
Team members were poised
Presentation and clearly articulated their
Skills thoughts. Every member
participated at a high level.
Speakers spoke loudly and
clearly, and maintained eye
contact. Presentation was
completed in the 10 minute
time allocation. Q&A were
completed in a coherent and
timely fashion.
FULTON
schools of engineering
Rubric
Rating 3
(Developing)
The presentation was mostly original,
clear, and complete. However not all the
speakers exhibited clearly that they
understood the design problem and that
their design met all the design
requirements.
The design details were clearly
presented. However, the audience had
some difficulty in following the design
evolution and/or understanding the final
design details. Not all slides were clear
and error-free.
Team members were mostly poised and
articulated their thoughts. Not all
members (a) participated at a high level,
(b) spoke loudly and clearly, and (c)
maintained eye contact. Presentation
ran slightly over the 10 minute time
allocation. All Q&A were not completed
in a coherent and timely fashion.
sustainable engineering and the built environment
Rating 1
(Beginning)
The presentation was difficult to
understand since it lacked coherence
and completeness. Most speakers
were unable to articulate an
understanding of the design problem
and were unable to show that their
design met all the design
requirements.
The design details were not clearly
presented. The audience had great
difficulty in following the design
evolution and/or understanding the
final design details. Slides contained
errors and exhibited a lack of clear
progression of ideas.
Team members were often inaudible
and relied heavily on their notes.
Speakers did not maintain eye contact
and engage the audience. Presentation
was completed too early or ran well
over the 10 minute time allocation.
Q&A were incomplete and lacked
coherence.