Massachusetts Department of Elementary and Secondary Education

75 Pleasant Street, Malden, Massachusetts 02148-4906 Main Telephone: (781) 338-3000 TTY: N.E.T. Relay 1-800-439-2370

EVALUATION CRITERIA SCORECARD

Agency Staff use only (This document will be used to evaluate all responses submitted to this Solicitation) Updated April 2013

Name of Bidder: Name of Reviewer(s): Date Reviewed:

Required Qualifications Criteria (Bidder must receive a “yes” on all the following criteria to move on to the next evaluation section)
Three (3) years experience in providing large-scale, high-stakes testing programs No Yes
Organizational capacity to deliver the program described in RFR at over 30 state testing centers by January 2014 No Yes
Scoring rubric: Outstanding: Application materials is complete, suggest exemplary qualifications, providing compelling, clear, and well-documented evidence of expertise. Excellent: Application materials are complete
and suggest solid qualifications, providing clear and well-documented evidence of expertise. Very Good: Application materials are complete and suggest adequate qualifications, providing clear evidence of expertise.
Good: Evidence of expertise or qualification in some areas is unclear or unsubstantiated by supporting documentation. Some required elements are missing. Satisfactory: Little evidence of qualification. Many required
elements are missing. Unsatisfactory: Qualifications is not applicable to request; elements missing / non-responsive to request.
Maximum
Points
Score Score Score Score Score Score Points
Awarded
Document Number: 14ACLAS1 Unsatisfactory Satisfactory Good Very Good Excellent Outstanding Multiplier
(Score 0–5 x
Available
0 pts 1 pt 2 pts 3 pts 4 pts 5 pts (5 x
Multiplier)
Multiplier)
Additional Evaluation Criteria:

Test Content and Forms (x2) 10

The bidder provides an overall description of
the assessment including subject area tests
and alignment with the CCSS, a description of
the item development process, and the use of
constructed-response items. The test
decription includes information on the numbers
and types of test forms available, the length of
subject-area tests, the availablity of the test in
languages other than English, and the
availability of alternate versions of test forms
for individuals who require certain
accommodations (e.g., Braille, large print,
audio, etc.).

Test Accommodations (x2) 10

The bidder addresses compliance with ADA
and includes a plan and process for providing
accommodations.The bidder describes the
information and application materials for
individuals with disabilities including
accommodation guidelines for advocates,
diagnosticians, and candidates for
accommodations. The bidder describes in
detail the available accommodatons for both
PBT and CBT. The bidder describes a review
and decision process for accommodation
eligibilty.

Test Delivery Systems (x5) 25

CBT -
The bidder includes a description of the
computer-based test and provides detailed
information about the required operating
system and software needs. The bidder
describes in detail the computer skills and
knowledge needed by the examinees.
The bidder describes how all the data from the
test administration platform will be gathered,
how the items will be scored and how the
scores will be summed to determin a passing
score. The bidder provides a time frame on
result turnaround including scoring the writing
requirement. The bidder describes how data
and test results will be transferred daily in an
encrypted format to the Department.
PBT -
The bidder describes in detail the roles of the
vendor, the test centers, and the Department in
the shipping, storage, and handling of secure
test materials for paper-based tests. The
bidder desctibes in detail the method by which
completed answer sheets will be scored,
including the roles of the vendor, the test
centers and the Department in the scoring
process. The bidder describes the materials
and equiptment that will be needed by the test
centers and/or the Department to perform their
roles in the scoring process. The bidder
describes the process and timetable by which
scores will be reported to the Examinees. The
bidder describes how data and test results will
be transferred daily in an encrypted format to
the Department.

Scoring and Reporting (x3) 15

The bidder describes how tests will be scored,
who will score constructed-response/writing
components of the test, how the test will be
normed, how the tests will be equated (linked
from year to year), and how the ESE can be
assured that the results of the test are a
reliable measure for determining whether to
award adults and out-of-school youth a
Certificate of High School Equivalency.

Preparation Materials, Training, and (x3) 15

Assistance
The bidder describes the availability of
preparation materials, including practice tests,
and any related professional development for
adult education programs. The bidder
describes training for testing center staff and
ESE staff regarding technical aspects of test
administration and accommodations and
assistance and support that will be available
related to testing issues. The bidder also
describes procedures and guidelines to ensure
 the integrity of the tests and the testing
process.

Security and Test Center Procedures (x3) 15

The bidder describes all procedures that will be
put in place to ensure the integrity of the tests
and the testing process, including but not
limited to the qualifications of test center
personnel, verification of the eligiblity and
identiy of examinees, test room supervision,
and recordkeeping. The bidder describes how
testing materials and confidential information
and data will be kept secure at all times. The
bidder must agree to adhere to all relevant
federal and state laws regarding the
confidentiality of information regarding
individuals and security protocols for handling
testing materials.

Costs/Budget (x5) 25
All costs associated with test development,
registration, administration, training, materials,
scoring, reporting, and any other program
activities, including retests, must be covered by
fees collected from examinees. The bidder
provides sufficient information in the budget to
show how the fee schedule will support testing
program costs. The bidder describes the
process of collecting the test fees and any
other proposed fees.

Preferred Qualification (x1) 5

The bidder has experience providing large-
scale, high-stakes testing programs for out-of-
school youth and/or adults. The bidder has
demonstrated the adherence of its tests and
testing programs to the AERA et al. (1999)
Standards for Educational and Psychological
Testing.

Supplier Diversity Program Plan meets the (x3) 15

guidelines of the program, provided letter
between vendor and partner, included copy of
approved letter from SDP office for partner.

Total: 135

Comments: