Rule: Measuring Educational Gain in The National Reporting System For Adult Education

Monday,
January 14, 2008
Part II
Department of
Education
34 CFR Part 462
Measuring Educational Gain in the
National Reporting System for Adult
Education; Final Rule
rmajette on PROD1PC64 with RULES2
VerDate Aug<31>2005 15:27 Jan 11, 2008 Jkt 214001 PO 00000 Frm 00001 Fmt 4717 Sfmt 4717 E:\FR\FM\14JAR2.SGM 14JAR2
2306 Federal Register / Vol. 73, No. 9 / Monday, January 14, 2008 / Rules and Regulations
DEPARTMENT OF EDUCATION On October 18, 2006, the Secretary a means for examining tests that are
published a notice of proposed currently approved for use in the NRS,
34 CFR Part 462 rulemaking (NPRM) for 34 CFR part 462 but that have not been updated recently
RIN 1830–ZA06 in the Federal Register (71 FR 61580). and, therefore, need to be reassessed for
In the preamble to the NPRM, the their continuing validity.
Measuring Educational Gain in the Secretary discussed on pages 61581 and
Analysis of Comments and Changes
National Reporting System for Adult 61582 the significant proposed
Education regulations. As a result of public In response to the Secretary’s
comment, these final regulations invitation in the NPRM, 13 parties
AGENCY: Office of Vocational and Adult contain several significant changes from submitted comments on the proposed
Education, Department of Education. the NPRM. While we fully explain these regulations. An analysis of the
ACTION: Final regulations. changes in the Analysis of Comments comments and of the changes in the
and Changes section elsewhere in these regulations since publication of the
SUMMARY: The Secretary establishes NPRM follows.
regulations, they are summarized as
procedures for determining the We group and discuss issues under
follows:
suitability of tests for use in the
• Rather than immediately the sections of the regulations to which
National Reporting System for Adult they pertain, with the appropriate
establishing, in § 462.4, a deadline for
Education (NRS). These final sections of the regulations referenced in
State and local eligible providers to stop
regulations also include procedures that parentheses. Generally, we do not
using tests that are currently listed in
States and local eligible providers must address technical and minor changes—
the Implementation Guidelines:
follow when using suitable tests for NRS and suggested changes the law does not
Measures and Methods for the National
reporting. authorize the Secretary to make.
Reporting System for Adult Education
DATES: These regulations are effective (Guidelines), the Secretary will
February 13, 2008. General Comment
announce a deadline in a notice
The incorporation by reference of published in the Federal Register after Comments: A commenter stated that,
certain publications listed in the rule is reviewing the first group of tests because each State uses its own
approved by the Director of the Federal submitted under these regulations. curriculum frameworks, the validity of
Register as of February 13, 2008. • On April 14, 2008, the Secretary a particular test may vary to the extent
However, affected parties do not have to will provide test publishers the first that the test aligns with a State’s
comply with the information collection opportunity to submit tests for review curricula. The commenter, therefore,
requirements in §§ 462.10, 462.11, under these final regulations. In stated that the Department could not
462.12, 462.13, and 462.14 until the subsequent years, in accordance with approve a test for use in all States
Department of Education publishes in § 462.10(b), test publishers must submit without evaluating its validity for each
the Federal Register the control number applications to the Secretary by October State that uses it.
assigned by the Office of Management 1 of each year. Discussion: We agree that not all
and Budget (OMB) to these information • We have revised several sections of States can use any single test. States are
collection requirements. Publication of the regulations to distinguish between expected to select a suitable test or tests
the control number notifies the public (1) traditional tests, which use items that best align with their particular
that OMB has approved these that have been generated before the test curricula. If a State’s curriculum is not
information collection requirements is administered, and (2) computerized aligned with an existing test, the State
under the Paperwork Reduction Act of tests, which use an algorithm to select will need to develop its own test aligned
1995. test items while the test is being with the State curriculum and submit
FOR FURTHER INFORMATION CONTACT: administered. The changes affect the test to the Department for review
Mike Dean, U.S. Department of §§ 462.3(b) regarding the definition of under these final regulations.
Education, 400 Maryland Avenue, SW., test, 462.11 regarding the information Changes: None.
Room 11152, Potomac Center Plaza, that must be included in a test Definitions
Washington, DC 20202–7240. publisher’s application, 462.12 and
Telephone: (202) 245–7828 or via 462.13 regarding the Secretary’s review Adult Education (§ 462.3)
Internet: Mike.Dean@ed.gov. of tests, and 462.41 regarding the Comments: A commenter stated that
If you use a telecommunications administration of tests. the proposed regulations incorrectly
device for the deaf (TDD), call the • Section 462.12(e) has been revised defined adult education. The
Federal Relay Service (FRS), toll free, at to clarify that test publishers can request commenter noted that the regulations
1–800–877–8339. that the Secretary reconsider a decision refer to students ‘‘who are not enrolled
Individuals with disabilities can to revoke a determination that a test is in secondary school’’ while the Act
obtain this document in an alternative suitable before the Secretary makes a refers to students ‘‘who are not enrolled
format (e.g., Braille, large print, final determination about the test’s or required to be enrolled in secondary
audiotape, or computer diskette) on suitability for measuring educational school under State law.’’ The
request to the contact person listed in gain for the NRS. commenter recommended using the
the preceding paragraph. Through these final regulations, we definition in the Act.
SUPPLEMENTARY INFORMATION: These formalize the process for the review and Discussion: Section 462.3(a) indicates
final regulations further the approval of tests for use in the NRS. We that certain terms used in the
Department’s implementation of section believe that the uniform process in these regulations, including adult education,
212 of the Adult Education and Family regulations will facilitate test are defined in section 203 of the Act.
Literacy Act (Act), 20 U.S.C. 9201 et publishers’ submissions of tests to the The language the commenter quotes is
seq., which establishes a system to Department for review and will help from the definition of adult education
assess the effectiveness of eligible strengthen the integrity of the NRS as a population in § 462.3(b), which is not
agencies in achieving continuous critical tool for measuring State defined in the Act. Nevertheless, we
improvement of adult education and performance on accountability agree that the two definitions should be
literacy activities. measures. This process also will provide consistent.
Federal Register / Vol. 73, No. 9 / Monday, January 14, 2008 / Rules and Regulations 2307
Changes: We have modified the review process, and have the legal right tests and (2) States and local eligible
definition of adult education population to submit the test for review. With recipients to transition from unsuitable
to include individuals who are not regard to the recommendation to have tests to suitable tests. We also intended
required to be enrolled in secondary the Secretary approve other entities who to impose a deadline that would result
school under State law in order to make can submit a test for review, it would be in the efficient removal of unsuitable
it consistent with the definition of adult inappropriate and counter-productive tests from use in the NRS. Once the
education in the Act. for the Secretary to determine the Secretary determines that a test is
suitability of a test submitted for review unsuitable for use in the NRS,
Content domains and skill areas
without the permission of the rightful permitting States to continue using it for
(§ 462.3)
owner of a registered copyright of the long periods of time would be
Comments: One commenter stated test or the entity licensed by the inconsistent with the Secretary’s intent
that the term skill areas should be used copyright holder to sell or distribute the to improve data quality.
consistently throughout the regulations, test. While we understand the desire to
instead of the regulations using this Changes: None. defer implementation of the regulations
term interchangeably with the terms because of cost factors and timing
content domain and content June 30, 2008, deadline for transitioning
constraints, improving the quality of
specifications. to suitable tests (§ 462.4)
State accountability systems and the
Discussion: In drafting the proposed Comments: Several commenters data reported by the NRS is of
regulations, we used the terms content expressed concern that States might not immediate importance and should not
domain and content specifications in have adequate time by the June 30, be unduly delayed. Adult Education
the sections of the regulations 2008, deadline to change assessment and Family Literacy Programs, like
applicable to test publishers because instruments, particularly if the Secretary other Federal programs, must report on
those are terms of art in the test determines that a test is no longer progress made, achievements, and
publishing industry. Likewise, the term suitable for use in the NRS. The overall program effectiveness using
NRS skill areas is used in the sections commenters stated that States need time valid and reliable measures of
of the regulations that are applicable to to rewrite assessment policies, select performance. The regulations are
States and local eligible recipients replacement tests, retrain personnel, designed to improve the reliability and
because this is a term of art in the adult purchase materials, modify complex validity of data used to report the
education program. Although we used data systems, and, possibly, hire special educational gains of students, and
the term content specifications in the contractors to assist with modifying thereby improve the reliability and
proposed regulations, we did not those data systems. A different validity of data on overall program
include it as a defined term. We think commenter stated that it might take two effectiveness.
it is appropriate to do so in the final years to implement a change in State In light of the commenters’ concerns,
regulations because the term has the assessment instruments. Commenters and to accommodate States’ needs to
same meaning as the terms content recommended that the regulations make system revisions, provide training,
domains and NRS skill areas. permit a State to negotiate a practical and acquire tests, we will not specify a
Changes: We have modified the transition timeline with the Secretary. date in these regulations by which
defined term, content domains or NRS Commenters also recommended that, States and local eligible providers must
skill areas, in proposed § 462.3 to also because transitioning from unsuitable cease using unsuitable tests; instead, we
include the term content specifications. tests places a burden on States’ financial have provided for the Secretary to
resources and professional development announce this deadline in a notice
Test publisher (§ 462.3) capabilities, the regulations should be published in the Federal Register.
Comments: A few commenters deferred until the amount of funds Changes: We have revised § 462.4 to
expressed concern that the definition of available for State leadership becomes provide that the Secretary will
test publisher might be too restrictive 15 percent of the Federal allocation. announce, through a notice in the
and could prevent the review of some One commenter indicated that local Federal Register, a deadline by which
tests. Commenters recommended programs often pay the significant cost States and local eligible providers must
expanding the definition to include of purchasing assessment instruments stop using tests that are currently listed
universities; adult education programs; and that replacing an entire assessment in the Guidelines and that the Secretary
other entities that possess sufficient system in a single budget year could has determined not to be suitable for use
expertise and capacity to develop, devastate a local budget. in the NRS under these final
document, and defend assessments; Discussion: Proposed § 462.4 would regulations.
entities in the process of copyrighting a have permitted States and local eligible
test; and entities holding an providers to continue to measure Deadline for submitting tests for review
unregistered copyright to a test. One educational gain using a test that was by the Secretary (§ 462.10(b))
commenter agreed that the Secretary identified in the Guidelines until June Comments: One commenter agreed
should review only tests from test 30, 2008. However, we specifically that the regulations should provide an
publishers owning a registered asked for comments on whether this annual deadline for test publishers to
copyright. deadline would provide sufficient time submit tests to the Secretary for review.
Discussion: The proposed regulations for States and local eligible recipients to Other commenters requested
did not prohibit universities, adult make the transition to suitable tests clarification on when the review cycle
education programs, or other legitimate because we recognized that changing begins and ends. Another commenter
test developers from submitting a test tests significantly affects a State’s asked if the first opportunity to submit
for review. We explained in the accountability system. Our intention in tests would be in 2007 or in 2008. Yet
preamble to the NPRM that entities proposing the June 30, 2008, deadline another commenter suggested that the
submitting tests for the Secretary’s was to ensure that States stop using date for submission of tests be no sooner
review must be knowledgeable about unsuitable tests on a date certain but than two months and no later than four
the test, be able to respond to technical still provide enough time for (1) the months after the effective date of the
questions the Secretary raises during the Secretary to complete one review of final regulations.
Discussion: We are establishing April require a test publisher to demonstrate such steps, the test publisher would not
14, 2008 as the first date by which test that adult educators have been involved be required to use any particular
publishers must submit tests for review in a test’s development and methodology for doing so.
under these regulations. In subsequent maintenance, and that some publishers Changes: None.
years, test publishers must submit would not meet that requirement easily.
Content of an application—Item
applications to the Secretary by October The commenter also stated that
development (§ 462.11(c))
1 of each year. However, because we compliance with the regulations would
cannot predict the number of tests that require customized tests developed Comments: A commenter noted that
will be submitted for review nor the specifically for use in adult education, the proposed regulations did not require
amount of time it will take to review the which would increase the cost and test publishers to include in their
tests, it is not possible to predict how exclude some quality assessments. applications information on item
long the process will take from year to Discussion: The regulations do not selection or form development for the
year. We, therefore, do not think it is require a test publisher to demonstrate test under review.
appropriate to establish a date on which that adult educators have been involved Discussion: The commenter’s
we will announce the results of the in a test’s development and observation is correct and calls attention
Secretary’s review. We will publish the maintenance. We realize that tests to the need for the regulations to require
list of suitable tests well before the developed for other populations might test publishers to include this
program year in which they might be not be suitable for use in the NRS information in their applications and for
used. because they were not developed with the regulations to clarify the distinction
Changes: None. the adult education population in mind between traditional tests, which use
and do not readily measure the items that have been generated before
Content of an application—General
educational functioning levels used in the test is administered, and those that
(§ 462.11)
the NRS. The regulations are clear that use a computerized algorithm to select
Comments: A commenter responded the Secretary reviews tests to determine test items while the test is being
positively to the regulations’ specific their suitability for use in the NRS. For administered.
delineation of what an application for instance, § 462.13(a) indicates that, in Changes: We added a new paragraph
test review must include. Another order for the Secretary to consider a test (3) to § 462.11(c) to require test
commenter asked whether test suitable for use in the NRS, the test publishers to describe in their
publishers must use a form in addition must measure the NRS educational applications the procedures used to
to submitting the information outlined functioning levels of members of the assign items (1) to forms, for tests that
in proposed § 462.11(b) through (j). adult education population. are constructed prior to being
Another commenter stated that it may Accordingly, § 462.11(c)(1)(ii) requires administered to examinees, or (2) to
be too constraining to require test information that demonstrates the examinees, for adaptive tests in which
publishers to arrange application extent to which the adult education items are selected in real time.
information in the order established by population was used to develop and
proposed § 462.11(b) through (j). evaluate a test, which is appropriate Content of an application—
Discussion: To facilitate the review because the tests will be used with that Maintenance: history of test use
process, the regulations in § 462.11 population. (§ 462.11(d)(4))
describe the specific requirements for Changes: None. Comments: A few commenters
the contents of an application. A test recommended that the regulations
Content of an application—Motivation
publisher is not required to submit any require test publishers to include in
of examinees (§ 462.11(c)(1)(iii))
form or information except as required their applications additional
in § 462.11. We believe that organizing Comments: Two commenters were information on the history of the test’s
the information in the application in the concerned that test publishers would use.
order presented in § 462.11(b) through have to include information in the Discussion: The regulations require
(j) will help to ensure that information application on the motivation of test publishers to provide
about a test is easily available to and examinees used in the development of documentation of how a test is
reviewable by the educational testing a test. One commenter indicated that maintained, including a history of the
and assessment experts who will review ‘‘there is no generally accepted method test’s use. We are particularly interested
the tests; however, to provide test for identifying and classifying the in information on how many times the
publishers with some flexibility in degree and level of motivation of test forms have been administered. This
organizing their applications, we will examinees.’’ The commenter stated that information is useful in gauging how
permit them to include in their a test publisher could make some much the test forms have been exposed
applications a table of contents that assumptions about motivation, but and the likelihood of test items being
identifies the location of the information indicated that the assumptions would compromised.
requested in § 462.11(b) through (j). be subjective and not scientifically Changes: We have revised
Changes: We have revised valid. The second commenter requested § 462.11(d)(4) to clarify that information
§ 462.11(a)(3)(ii) to permit test clarification of the expectation that submitted in the application regarding
publishers to include a table of contents examinees would be motivated. the history of a test’s use must include
in their applications as an alternative to Discussion: The regulations only information on the number of times the
presenting information in the require test publishers to provide in test has been administered.
application in the order described in their applications information on the
steps, if any, taken to ensure that Content of an application—Maintenance
§ 462.11(b) through (j).
examinees were motivated while (§ 462.11(d)(5))
Content of an application—Involvement responding to the test. The regulations Comments: A commenter

of the adult education population do not require test publishers to take recommended that the regulations
(§ 462.11) steps to ensure that examinees were require test publishers to include in
Comments: A commenter stated that motivated while responding to the test. their applications the procedures used
proposed § 462.11 would generally Further, if a test publisher were to take for computerized adaptive tests to select
subsets of items for administration, standard-setting procedures in the 1999 publisher providing evidence that the
determine the starting point and edition of the Standards for Educational transformation rules result in reliable,
termination conditions, score the tests, and Psychological Testing and will help i.e., consistent, educational functioning
and control item exposure. test publishers demonstrate the level classifications. We believe that,
Discussion: We agree that requiring suitability of their tests for measuring when an application is submitted, a test
test publishers to provide the educational gain for the NRS. publisher should be able to provide
recommended information will help Test scores are only useful in this documentation of the degree of
experts to better assess the suitability of context if they can accurately classify consistency in performance across
computerized adaptive tests for use in individuals according to NRS levels. different forms of the test, particularly
the NRS. Therefore, it is necessary for test regarding which examinees are
Changes: We added a new paragraph publishers to demonstrate how the classified into the same NRS
(5) to § 462.11(d) to require test range of test scores map onto the NRS educational functioning levels across
publishers to include in their levels and do so in a reliable and valid different forms of the test. By
applications for computerized adaptive fashion. In the test development demonstrating that a test can
tests the information recommended by process, developers need to show that consistently classify individuals into the
the commenter. the range of test scores produced on same NRS educational functioning
their tests covers the range of skills levels across different forms of the test,
Content of an application—Match of
depicted in the NRS levels, and more the test publisher assures the
content to the NRS educational
importantly, shows which range of Department that assessments of
functioning levels (content validity)
scores corresponds to a specific NRS educational gain are the result of
(§ 462.11(e)(2) and (4))
level. instruction and other interventions to
Comments: A few commenters asked Changes: None. improve literacy, not measurement
if proposed § 462.11(e)(2) and (4) were error. Without this demonstration of
requesting the same information, and Content of an application—Reliability
(§ 462.11(g)) classification consistency, reports of
sought clarification regarding the educational gain are uninterpretable.
difference between the paragraphs. Comments: A commenter noted that This information is very important to
Discussion: The paragraphs are in discussing reliability the proposed determinations about the suitability of a
requesting the same information. regulations used the phrase ‘‘the test and whether the test measures the
Changes: We have removed correlation between raw or number NRS educational functioning levels as
§ 462.11(e)(4) to eliminate the duplicate correct scores.’’ The commenter noted required in § 462.13(a).
information requirement and that this phraseology is not applicable to Changes: None.
renumbered the remaining paragraphs. tests that use an adaptive structure or a
multi-parameter item response theory Content of an application—Construct
Content of an application—Procedures validity (§ 462.11(h))
for matching scores to NRS educational model. The commenter stated that, in
such situations, the particular items Comments: A commenter expressed
functioning levels (§ 462.11(f)(2)) concern that proposed § 462.11(h)
answered correctly, not the number of
Comments: A commenter stated that items answered correctly, determine the would have required the results of
requiring the judgments of subject- score. several studies on the adult education
matter experts to translate an Discussion: We agree with the population in connection with other
examinee’s performance to the commenter that the phrase in the tests designed to assess educational
examinee’s standing with respect to the regulations is not applicable to gain, which can be useful and
NRS educational functioning levels computerized adaptive tests. meaningful, but also time-consuming
might not prove fruitful and would Changes: We revised § 462.11(g)(1) to and expensive. The commenter
substantially increase the cost of test require that, in the case of computerized indicated that imposing this
development. The commenter stated adaptive tests, test publishers document requirement after, not before, test
that determination of score ranges and in their applications the correlation approval would permit test publishers
their fit to the existing NRS levels can between raw (or scale) scores across to collaborate and conduct the studies
be made based on an analysis of skills alternate administrations of the test. in a more cost-effective manner.
being assessed and an intimate Comments: With regard to proposed Further, the commenter stated that the
knowledge of the assessment tools being § 462.11(g)(2), a commenter suggested requirement could exclude some
used. that information about the number of qualified assessments. The commenter
Discussion: Section 462.11(f) does not individuals classified into NRS levels recommended that the regulations be
require the use of subject matter experts. would only provide useful data if the rewritten so that (1) these studies would
It requires test publishers to document information were submitted after the only be required of tests that have been
the procedure they use to translate the Department approved a test’s scores-to- approved and (2) a five-year period
performance of an examinee to the NRS-levels crosswalk. The commenter could be provided for conducting the
examinee’s standing with respect to the stated that requiring this information studies.
NRS educational functioning levels. A prior to test approval could produce Discussion: We do not agree that a test
test publisher can choose the procedure information that is not meaningful. should be approved for use in the NRS
it thinks is best. However, if a test Another commenter responded prior to the test publisher providing
publisher chooses to use judgment- positively to the requirement for documentation of the appropriateness of
based procedures to translate ‘‘inclusion of information about a test for measuring educational gain for
performance, the regulations require the decision/classification consistency.’’ the NRS, i.e., documentation that the
publisher to provide information on that Discussion: We do not agree that the test measures what it is intended to
procedure, including information Department should approve the rules a measure. Section 462.11(h) is consistent
concerning the subject matter experts test publisher uses to transform the with the 1999 edition of the Standards
the test publisher used. Requiring this scores of a test into estimates of for Educational and Psychological
information is consistent with accepted examinees’ NRS educational Testing, which stresses the importance
professional test development and functioning levels prior to the test of gathering evidence of the test’s
construct validity. The Secretary cannot Content of an application—Other whether a test is suitable for use in the
determine whether a test is suitable for information (§ 462.11(i)(1)) NRS, we recognize that the Secretary
use in the NRS without having evidence Comments: A commenter requested can require a test publisher to provide
of the test’s construct validity. clarification of the phrase ‘‘an analysis additional information to establish a
Changes: None. of the effects of time on performance’’ test’s compliance with the criteria and
requirements in § 462.13. Section
used in proposed § 462.11(i)(1). The
Content of an application—Construct 462.11(i)(6) merely alerts test publishers
commenter thought the phrase meant
validity (§ 462.11(h)(1)) to this possibility.
‘‘the effects of the time it takes to Changes: None.
Comments: A commenter expressed administer the test or for a student to
concern that proposed § 462.11(h)(1) complete it.’’ Content of an application—Previous
would have required test publishers to Discussion: The commenter is correct. tests (§ 462.11(j))
document the appropriateness of a given Section 462.11(i)(1) requires an Comments: A commenter asked if a
test for measuring educational gain in application to include a description of test publisher submitting an application
the NRS, including the correlation the manner in which test administration for a test that is currently approved for
between the NRS test results and the time was determined and an analysis of use in the NRS would have to provide
results of other tests that assess the speededness of the test. The term the information in proposed § 462.11(b)
‘‘speededness’’ as used in § 462.11(i)(1) through (i) or would have to provide
educational gain in the same adult
refers to the effects on test performance only documentation of periodic review
education population. The commenter
that result from the time it takes to of the content and specifications of the
stated that this comparison could lead
administer or to complete the test. test as specified in proposed
to faulty conclusions if the other test is
Changes: We have revised § 462.11(j)(1).
not an accurate measure of the
§ 462.11(i)(1) to clarify that we require Discussion: As indicated in
construct. both information on the manner in § 462.11(a)(1), a test publisher must
Discussion: We believe that it is which test administration time was include with its application information
appropriate to look at test correlations determined and an analysis of the listed in paragraphs (b) through (i) as
as one criterion for evaluating construct speededness of the test. well as the applicable information in
validity. Generally, a test should paragraph (j). All applications must,
Content of an application—Other
correlate with other tests known to therefore, include the information in
information (§ 462.11(i)(5))
measure the same construct, and it § 462.11(b) through (i).
should not correlate (or have a very low Comments: A commenter requested Changes: None.
correlation) with tests known to clarification of the term ‘‘retesting Comments: A commenter requested
measure different constructs. This latter procedures’’ used in proposed clarification of the term ‘‘periodic
relationship depends upon the nature of § 462.11(i)(5). The commenter asked if review’’ used in proposed § 462.11(j)(1)
the comparison constructs. To the the term referred to ‘‘pre- and post- and (2) with regard to the
extent that the two constructs are testing.’’ documentation that must be submitted
theoretically related, a correlation that Discussion: The term ‘‘retesting’’ as for a previous test to ensure that it
approximates their theoretical used in § 462.11(i)(5) refers to the re- continues to reflect NRS educational
relationship is expected. administration of a test that might be functioning levels. The commenter
necessary because of problems in the asked if annual or bi-annual reviews
Changes: None. original administration (e.g., the test constitute a ‘‘periodic review,’’ and
Content of an application—Construct taker becomes ill during the test and suggested that the regulations use the
validity (§ 462.11(h)(2)) cannot finish, there are external term ‘‘current review’’ and specify the
interruptions during testing, or there are intervals for the reviews.
Comments: Two commenters stated administration errors). The commenter also suggested that
that proposed § 462.11(h)(2) should be Changes: We have revised the regulations address the possibility
reconsidered because ‘‘hours of § 462.11(i)(5) to clarify that ‘‘retesting’’ that the NRS educational functioning
instruction’’ is not a variable that refers to the re-administration of a test levels might change. The commenter
correlates highly with test scores. that might be necessary because of stated that realigning tests to revised
Discussion: Proposed § 462.11(h)(2) problems in the original administration NRS educational functioning levels
would have required test publishers to such as, the test taker becomes ill during would cause a burden for test
document that a test measures what it the test and cannot finish, there are publishers, and that test publishers need
is intended to measure, including the external interruptions during testing, or at least one year advance notice of any
extent to which the raw or scale scores there are administration errors. proposed changes in the NRS
are related to other relevant variables, educational functioning levels.
Content of an application—Other Discussion: Section 462.11(j) requires
such as hours of instruction or other information (§ 462.11(i)(6)) test publishers to provide specific
important process or outcome variables.
Comments: A commenter noted that information about currently used tests
While we are aware of data establishing proposed § 462.11(i)(6) would require to ensure that the tests continue to
the relationship between Adult Basic test publishers to provide such other reflect NRS educational functioning
Education (ABE) test scores and hours evidence as the Secretary determines is levels. The shorter the period of time
of instruction, the reference to ‘‘hours of necessary to establish the test’s between reviews of a test, the more
instruction’’ in § 462.11(h)(2) was only compliance with the criteria and relevant the results would be in
intended to provide an example of a requirements in proposed § 462.13. The determining the test’s content validity
possibly relevant variable. commenter requested clarification with regard to NRS educational
Changes: We have revised regarding what that evidence would be. functioning levels. However, we are
§ 462.11(h)(2) to clarify that ‘‘hours of Discussion: While § 462.11 includes reluctant to specify a time-frame for
instruction’’ is an example of possibly the information we anticipate the reviews because we do not want to
relevant variables. Secretary will need to determine create an additional burden for test
publishers by requiring reviews more algorithm, such as computerized Procedures the Secretary uses to review
frequently than a test publisher adaptive tests, multistage adaptive tests, the suitability of tests (§ 462.12(b))
typically would perform. or linear on-the-fly tests.’’ The Comments: A commenter asked if the
With regard to providing advance commenter suggested specifically results of the Secretary’s review of a test
notice to test publishers about changes changing the proposed regulatory would be posted for public review.
in the NRS educational functioning language from ‘‘and have two or more Another commenter suggested that the
levels, in the past we have customarily secure, parallel, equated forms’’ to ‘‘and regulations should provide that test
provided notice to all concerned parties, have two or more secure, parallel forms publishers be notified at least 30 days
including test publishers, well in if the test is produced before it is before the Secretary notifies States and
advance of any changes to the NRS administered to a learner (e.g., paper- local eligible providers that a test is
educational functioning levels by based tests). If the test uses a unsuitable for use in the NRS. With
posting notices on the Internet at computerized algorithm for regard to tests that the Secretary
http://www.nrsweb.org. We intend to administering items in real time, the determines are unsuitable, a different
continue that practice and, because the
size of the item pool and the method of commenter recommended that the
educational functioning levels are
item selection must ensure negligible regulations provide a time-frame by
included in § 462.44 of these final
overlap in items across pre- and post- which the Secretary would review any
regulations, will publish a notice of
test.’’ Another commenter also additional information submitted by the
proposed rulemaking in the Federal
emphasized the importance of test publisher and make a final
Register requesting comment on any
recognizing computerized tests. determination.
proposed changes to § 462.44. Discussion: In accordance with
Changes: None. Discussion: As we have discussed
Comments: A commenter asked if the § 462.12(c)(2), the Secretary will
elsewhere in this notice, we agree that
phrase ‘‘previous tests used in the NRS’’ annually notify the public through a
the regulations should clarify the notice published in the Federal Register
in proposed § 462.11(j)(1) referred only distinction between (1) traditional tests,
to those tests previously listed in the and posted on the Internet at http://
which use items that are generated www.nrsweb.org of tests that are suitable
NRS Implementation Guidelines. before the test is administered, and (2)
Discussion: The ‘‘previous tests’’ for use in the NRS. Under § 462.12(e)(5),
computerized tests, which use an the Secretary will follow the same
discussed in § 462.11(j)(1) are tests that algorithm to select test items while the
were listed in the Guidelines and used procedure to notify the public when a
test is being administered. test that was previously determined
to measure educational gain in the NRS
before the effective date of these final Changes: We have revised §§ 462.3 suitable is determined to be unsuitable.
regulations. (the definition of test), 462.11(c)(3), However, the Secretary will not post for
Changes: None. 462.12(a)(2)(iii), 462.13(e), and public review the Secretary’s
Comments: A commenter requested 462.41(c)(3) to provide the clarification determination regarding a test that has
clarification of the requirement in recommended by the commenters. not been previously reviewed and that
proposed § 462.11(j)(3) for test the Secretary determines to be
publishers to submit ‘‘new data’’ for Tests that measure some, but not all, unsuitable. We do not believe this is
tests that have not changed in the seven educational functioning levels necessary because the Secretary’s
years since the Secretary determined the (§§ 462.12(a)(2)(iv) and 462.13(b)) determination regarding these tests will
tests were suitable for use in the NRS. not have an impact on the public,
Comments: A commenter requested
Discussion: The intent of § 462.11(j) is States, or local eligible providers.
clarification on whether the Secretary Proposed § 462.12(d) provided that a
to ensure that a test publisher provides
will review tests that currently measure test publisher would have 30 days to
specific information about a test that has
only some, but not all, educational request that the Secretary reconsider the
been in use for years, but that has not
changed in the seven years since the functioning levels. decision that a test is unsuitable.
Secretary determined the test was Discussion: The regulations provide Therefore, it will be after this 30-day
suitable for use in the NRS. In this for the Secretary to review tests that period that the Secretary will notify the
circumstance, the regulations require measure some, but not all, educational public of a decision regarding a test.
test publishers to provide new, i.e., functioning levels. Sections Because it is impossible to anticipate
updated, data that support the validity 462.12(a)(2)(iv) and 462.13(b) indicate the complexities that might be involved
of the test. that the Secretary reviews and in decisions regarding a test that was
Changes: We have revised determines the suitability of a test if an initially determined to be unsuitable,
§ 462.11(j)(3) to clarify that test application includes a test that samples we do not believe it would be
publishers must provide updated data to one or more of the major content appropriate to limit the time the
support test validity. domains of the NRS educational Secretary takes to review additional
functioning levels of Adult Basic information provided by a test publisher
Computerized tests (§§ 462.3, and make a final determination
462.11(c)(3), 462.12(a)(2)(iii), 462.13(e), Education (ABE), Adult Secondary
Education (ASE), or English-As-A- regarding the suitability of a test.
and 462.41(c)(3)) However, we intend to conduct the
Second Language (ESL) with sufficient
Comments: A commenter review as expeditiously as possible.
numbers of questions to represent
acknowledged that the proposed Changes: None.
regulations attempted to distinguish adequately the domain or domains.
between traditional paper-based tests Further, § 462.12(b)(2) provides Publishing the list of suitable tests
and computerized tests. However, the flexibility for the Secretary to determine (§ 462.12(c))
commenter recommended that the that a test or a sub-test is suitable for use Comments: A commenter suggested
regulations be modified to clarify that in the NRS if it measures the content that if tests must be submitted for
the term ‘‘parallel forms’’ refers to domain of some, but not all, of the NRS review by October 1 of each year, the
‘‘paper-based tests or computerized tests educational functioning levels. test publisher and adult education
that do not involve an item selection Changes: None. programs should be informed of which
tests are approved no later than content specifications, item types or test-taking, the regulations use the term
February 1 of the following year. sub-tests. ‘‘publisher’s recommendations’’ instead
Discussion: Within a reasonable time- The proposed regulations were, of the term ‘‘publisher’s guidelines’’ that
frame, the Secretary will review tests, therefore, unclear as to whether the was used in proposed §§ 462.13(d) and
notify test publishers of the results of Secretary could revoke a determination 462.40(c)(3)(ii).
the review, and provide States and local about the suitability of a test if the test Discussion: We do not agree that the
eligible providers with a list of suitable had been substantially revised. We term ‘‘publisher’s guidelines’’
tests. However, because we cannot intended that revision of a test would be emphasizes the use of a single protocol
predict the number of tests that will be a valid reason for revoking a when determining when to administer a
submitted for review nor the amount of determination about test suitability. post-test. State and local eligible
time it will take to review the tests, it Changes: We have revised § 462.12(e) providers, like the commenter, are
is not possible to predict how long the to clarify that substantial changes to a aware that many factors influence the
process will take from year to year. We, test is one of the reasons the Secretary appropriate time between pre-testing
therefore, do not think it is appropriate can revoke a determination regarding and post-testing. These factors are taken
to establish a date by which we will the suitability of a test. into consideration by test administrators
announce the results of the Secretary’s Criteria and requirements for at the local level. However, because
review. We will publish the list of determining test suitability tests differ, test administrators rely on
suitable tests well before the program (§ 462.13(c)(1), (d), and (f)(3) (proposed the guidelines, developed during test
year in which they can be used. (g)(3)) construction and validation and
Changes: None. provided by test publishers, to help
Comments: A commenter noted that ensure that a sufficient amount of time
Revocation of determination that a test The Standards for Educational and has passed before a post-test is given in
is suitable (§ 462.12(e)) Psychological Testing referenced in order to optimize the measurement of
§ 462.13(c) provides specific guidelines educational gains.
Comments: A commenter suggested for the technical documentation,
that test publishers be notified of the Changes: None.
including evidence of validity that Comments: A commenter requested
Secretary’s decision to revoke a should be made available to interested clarification of § 462.13(f)(3) (proposed
determination that a test is suitable parties. The commenter recommended § 462.13(g)(3)), which requires the
before the Secretary notifies the general clarifying that publishers should publisher of a test modified for an
public. The commenter stated that test provide the Secretary with information individual with a disability to
publishers should be given sufficient regarding test development and validity. recommend educational functioning
time to address the Secretary’s concerns Discussion: In proposed levels based on the previous
before the Secretary revokes the § 462.13(c)(1), we indicated that, in performance of test-takers who are
determination that the test is suitable. order for a test to be determined suitable members of the adult education
This would provide a process for use in the NRS, the test would have population of interest to the NRS in
comparable to the process proposed for to meet all applicable and feasible order for the Secretary to consider a test
a test that the Secretary determines is standards for test construction provided suitable for use.
unsuitable. in The Standards for Educational and Discussion: In order for the Secretary
Discussion: We agree that test Psychological Testing. We did not to consider a test that has been modified
publishers should have an opportunity intend this language to imply that the for individuals with disabilities suitable
to address the Secretary’s decision to Secretary would review only for use in the NRS, test publishers must
revoke a determination that a test is information that is related to test (1) demonstrate that adult education
suitable before that determination development. We agree with the students with disabilities were included
becomes a final decision. commenter that validity is an important in the pilot or field test referred to in
Changes: We have revised § 462.12(e) factor in test evaluation and make it § 462.11(c)(1); (2) match scores to the
to give test publishers an opportunity to clear in § 462.11(e) through (g), that test NRS educational functioning levels
request that the Secretary reconsider a publishers must include in their based on the information obtained from
decision to revoke a determination that applications information on content adult education students with the
a test is unsuitable. validity, reliability, and construct disability who participated in the pilot
Comments: A commenter stated that validity. However, we think adding the or field test and for whom the test has
proposed § 462.14(b) was unclear reference to ‘‘validity’’ in § 462.13(c)(1) been adapted; and (3) provide in the
concerning the reasons a test’s seven- will reinforce the importance of test application, as required in § 462.11(f),
year approval status might be revoked. publishers providing information on documentation of the adequacy of the
Discussion: Section 462.12(e), not validity. procedure used to translate the
§ 462.14(b), establishes the reasons for Changes: We have revised performance of adult education students
which the Secretary can revoke the § 462.13(c)(1) to require a test to meet all with the disability for whom the test has
determination regarding the suitability applicable and feasible standards for been modified to an estimate of the
of a test. In the proposed regulation, we test construction and validity that are examinees’ standing with respect to the
stated that the Secretary can revoke the provided in the Standards for NRS educational functional levels.
determination if the Secretary Educational and Psychological Testing. Changes: In response to the comment,
determines that the information Comments: A commenter noted that we changed § 462.13(f)(3) to clarify that
submitted as a basis for the Secretary’s many factors influence the appropriate the Secretary considers a test modified
review of the test was inaccurate. In time between testing and retesting, for individuals with disabilities suitable
proposed § 462.14(b), however, we including intensity of instruction, for use in the NRS if test publishers (1)
implied that the Secretary also could frequency and length of class meetings, recommend educational functioning
revoke a determination regarding the and class size. The commenter wanted levels based on the information
suitability of a test if the test is to ‘‘decrease the emphasis on a single obtained from adult education students
substantially revised—for example, by protocol for post-testing’’ and suggested who participated in the pilot or field
changing its structure, number of items, that, when discussing the time between test and who have the disability for
which the test has been adapted and (2) Meaning of the term ‘‘placement’’ [functioning] level descriptors need to
provide documentation of the adequacy (§§ 462.40(c)(4), (c)(6) and (c)(10), be used to place students, but the skill
of the procedure used to translate the 462.41(c)(3) (proposed), 462.42(a), used should be the areas most relevant
performance of adult education students (b)(2), and (c)(1), (d)(1), and (d)(2), and to the students’ needs and the program’s
with the disability for whom the test has 462.43(a)(2) and (b)) curriculum. If multiple skill areas are
been modified to an estimate of the Comments: A few commenters stated assessed and the student has differing
examinees’ standing with respect to the that using the term ‘‘placement’’ in the abilities in each area, however, NRS
NRS educational functional levels. policy requires that the program place
regulations to mean the ‘‘assignment of
the student according to the lowest skill
a student to the appropriate NRS
Subpart D—General (§§ 462.40–462.44) area.’’ The Department’s policy ensures
educational functioning level’’ is
that States use a standardized approach
Comments: Several commenters noted confusing because the traditional
when reporting educational gain, which
that the Department currently provides meaning of placement is ‘‘placing a
ensures comparability of data. These
non-regulatory guidance to States on the student in a particular class or other
regulations use the term ‘‘placement’’ as
NRS. The commenters asked why the instructional offering.’’ Commenters
a term of art to refer to the placement
Department is issuing regulations when noted that tests designed for placement
of students in NRS educational
most States comply with and are are different from those used to measure
functioning levels in order to measure
successfully implementing the non- educational gain. and then report on educational gain in
Discussion: The regulations, like the
regulatory guidance and the guidance the NRS. Placement of a student in an
Guidelines currently used by States, use educational functioning level for NRS
provides each State with needed
the terms ‘‘place’’ and ‘‘placement’’ as purposes does not affect placement in
flexibility. The commenters stated that terms of art to refer to the placement of
‘‘as long as States are complying with an instructional program. States can use
students in NRS educational a variety of tools when devising an
guidelines and meeting performance functioning levels in order to measure
standards, we understand that the instructional program for students.
and then report on educational gain in States and local eligible providers know
Federal role is to limit regulations and the NRS. It is only within this context
allow the States to continually increase that using the lowest test score is a
that the terms are used, and no other convention used for reporting purposes
their capabilities to improve program meaning should be inferred. and it is not intended to encourage
services.’’ The commenters, therefore, Changes: None. programs to teach students only in skill
recommended that subpart D be
Placing students (§ 462.42(d)(1) and (2)) areas in which students scored the
removed from the final regulations and lowest on tests or only in skill areas in
that the Secretary provide technical Comments: A commenter stated that
which gain is expected. The policy also
assistance and resources to the few placing a student in an NRS educational
is not intended to restrict the number or
States that are having difficulty creating functioning level using the lowest test type of assessments used to identify a
effective assessment and data score could result in programs student’s needs and to customize an
procedures. Additionally, one of these providing instruction in skill areas that instructional program to meet those
commenters stated that while guidance are not most relevant to the student. needs. In fact, programs are expected to
is valued, regulations sometimes seem Further, the commenter stated that teach students in multiple areas
arbitrary, intrusive, and unnecessary. programs (1) should not place students depending on the students’ needs and
based on one test and (2) should be the programs’ curricula.
Discussion: It is the Department’s allowed to use multiple placement tools
policy to regulate only when essential to We do not agree with the commenter
and other pertinent information, such as that using the lowest test score might
promote quality, and then in the most student goals, to determine placement. introduce systematic, rather than
flexible, most equitable, and least Another commenter stated that all random, measurement error. Using the
burdensome way possible. We believe scores have some measurement error lowest test score in order to
these regulations comply with that and that instituting a policy of using the operationalize educational gain in a
policy. While the regulations in subpart lowest test score might introduce consistent manner across States for
D are legally binding, they largely codify systematic, rather than random, reporting purposes is error free. If tests
the guidance provided in the Guidelines measurement errors. This commenter have met the standards established in
and the ‘‘State Assessment Policy also indicated that the policy of using these regulations, they should be able to
Guidance.’’ Although States have made the lowest test score could encourage assess skills at any level with minimum
significant progress, there remains programs to teach students only in an error.
variability in the quality of State area where gain is to be expected. The Changes: None.
processes and procedures in the commenter stated that a better policy
would be either to focus on the learner’s Measuring educational gain (§ 462.43)
collection and reporting of data on
student assessment and performance. primary subject area at pre-test, or Comments: A commenter suggested
These regulations, like the Guidelines, evaluate gain based on a composite the Department measure significant gain
technical assistance activities, and other score across areas. as determined by test developers. The
Discussion: With regard to placing a commenter thought this approach
efforts the Department has supported to
student using the lowest test score when would be the most accurate measure of
improve data quality, provide a
the student is tested in multiple skill gain. The commenter opposed the
significant tool to create a standard level
areas, § 462.42(d)(1) and (2) are approach in proposed § 462.43, stating
of quality among all States and thereby consistent with the policy in the that it would only capture learning gain
strengthen the integrity of the NRS as a Guidelines, which indicates: ‘‘States when a learner completes an
critical tool for measuring State should provide to local programs the educational functioning level by
performance and the impact of adult criteria for placing students at each crossing artificial cut points regardless
education. educational functioning level, using test of the starting level. Another commenter
Changes: None. scores from the initial assessment. Not stated that the approach for measuring
all of the skill areas described in the gain in § 462.43 is ‘‘too coarse, is likely
to capture only extreme gain, and will instruction should now complete their in an educational functioning level for
miss very significant educational gains preparation by moving from ESL classes NRS purposes does not affect placement
that occur within an EFL [educational to either adult basic education or adult in an instructional program. A student
functioning level].’’ The commenter secondary education classes. The who scores above an SPL 6 might still
suggested that the Department consider commenters expressed their belief that be served in adult education programs
using other options for demonstrating OVAE’s suggestion is pedagogically if he or she has an educational need
educational gain such as ‘‘achieving a problematic and, more importantly, below the postsecondary level. States
gain score that is beyond chance would prevent programs from have the flexibility to determine how
expectations (using a scale score, which adequately addressing the learning programs are structured, how students
is more reliable than a proficiency needs of ESL learners. Further, the are placed in programs, and how
classification).’’ Another commenter commenters opposed stopping ESL instruction is delivered.
stated that ‘‘awarding only one instruction at the ‘‘Advanced ESL’’ Changes: None.
educational gain to students completing literacy level or SPL 6 because they
more than one educational functioning believe students at that level do not Student Performance Levels (SPL)
level to a high degree misrepresents and have language and literacy skills that are (§ 462.44)
under reports the effectiveness and sufficient to transition successfully to Comments: A commenter requested
accomplishment of both adult teacher postsecondary education or to meet the clarification of the relationship between
and learner.’’ This commenter also demands of the workplace—two of the the NRS literacy levels and SPLs in the
stated that the current NRS policy of purposes for adult education that are Functioning Level Table. The
reporting an educational gain for cited in the Act. Commenters noted that commenter indicated that no exact
obtaining a General Educational the regulations do not place this same definition of SPL levels is provided and,
Development (GED) diploma by learners restriction on native English speakers therefore, their relationship to the NRS
in ASE II, but not to those in ABE or and recommended that the ‘‘High levels is unclear.
ASE I, misrepresents and under-reports Advanced ESL’’ literacy level be Discussion: The SPLs were developed
accomplishments. A different restored to the Functioning Level Table. by the Center for Applied Linguistics to
commenter suggested that attainment of Finally, the commenters recommended provide a standard description of adult
a GED diploma should be recognized as that the regulations provide States with refugees’ abilities at a range of levels
an educational gain. flexibility and not limit program and a common standard for ESL level
Discussion: The approach for services. descriptions for use by programs
measuring educational gain in § 462.43 Discussion: Commenters are correct in nationwide. They are nationally
is well established and accepted by the that proposed § 462.44 would codify a recognized in the adult ESL education
field. Over a two-year period, the change in the Guidelines by eliminating
community and represent a standard
Department consulted with States and the ‘‘High Advanced ESL’’ literacy level
metric for identifying skill levels of
convened a statutorily mandated panel from the Functioning Level Table. The
adult ESL students in general language
in order to develop a performance change was made in 2006 as part of the
ability, listening comprehension, and
accountability system for the adult information collection request for the
oral communication. The Functioning
education program. During that time, Guidelines. However, before the change
Level Table provides educational
consensus was reached on defining the was made, (1) OVAE consulted with
functioning level descriptors for
performance measures, including how Adult Education State Directors and (2)
students at literacy levels in ABE, ASE,
to measure educational gain. States and the Office of Management and Budget
and ESL. The literacy levels are divided
other participating entities agreed that (OMB) provided interested Federal
the approach in § 462.43 is the most agencies and the public, including into equivalent grade levels for ABE and
effective means of obtaining accurate, States and local eligible providers, an ASE and into SPLs for ESL. The
reliable, and valid information on opportunity to comment on the descriptors illustrate the types of skills
student performance. information collection request, which students functioning at a given level are
Changes: None. included the removal of the ‘‘High likely to have. The descriptors do not
Advanced ESL’’ literacy level from the provide a complete or comprehensive
High Advanced ESL (§ 462.44) delineation of all the skills at a
Functioning Level Table. We received
Comments: Some commenters no negative comments regarding our particular level, but instead, provide
opposed proposed § 462.44 because it, intent to eliminate the ‘‘High Advanced examples to guide assessment and
like the Guidelines, eliminated the ESL’’ literacy level, and therefore, instruction.
‘‘High Advanced ESL’’ literacy level changed the Functioning Level Table in Changes: None.
from the Functioning Level Table. The the Guidelines and in the proposed Executive Order 12866
commenters noted that the change regulations.
means that State and local agencies will The change in the Functioning Level We have reviewed these final
no longer be able to report educational Table does not mean that adult regulations in accordance with
gain for adult English language learners education programs can no longer Executive Order 12866. Under the terms
above the ‘‘Advanced ESL’’ literacy provide services to ‘‘High Advanced of the order we have assessed the
level or Student Performance Level ESL’’ students. These regulations are potential costs and benefits of this
(SPL) 6. The commenters stated that, consistent with the Act, which regulatory action.
based on communications with the authorizes services below the The potential costs associated with
Department’s Office of Vocational and postsecondary level, and do not change the final regulations are those resulting
Adult Education (OVAE), the change who can be served by adult education from statutory requirements and those
also means that programs can no longer programs. The educational functioning we have determined to be necessary for
provide ‘‘High Advanced ESL’’ levels largely serve to classify students administering this program effectively
instruction in adult education programs. for reporting purposes and should not and efficiently.
The commenters indicated that OVAE be viewed as the standard for In assessing the potential costs and
has suggested that students who would establishing the type of instruction that benefits—both quantitative and
have received ‘‘High Advanced ESL’’ can be provided. Placement of a student qualitative—of these final regulations,
we have determined that the benefits of To use PDF you must have Adobe 462.42 How are tests used to place students
the regulations justify the costs. Acrobat Reader, which is available free at an NRS educational functioning level?
We have also determined that this at this site. If you have questions about 462.43 How is educational gain measured?
regulatory action does not unduly using PDF, call the U.S. Government 462.44 Which educational functioning
levels must States and local eligible
interfere with State, local, and tribal Printing Office (GPO), toll free, at 1– providers use to measure and report
governments in the exercise of their 888–293–6498; or in the Washington, educational gain in the NRS?
governmental functions. DC, area at (202) 512–1530.
We discussed the potential costs and Authority: 20 U.S.C. 9212, unless
Note: The official version of this document otherwise noted.
benefits of these final regulations in the is the document published in the Federal
preamble to the NPRM under the Register. Free Internet access to the official Subpart A—General
headings Significant Proposed edition of the Federal Register and the Code
Regulations (pages 61581 and 61582), of Federal Regulations is available on GPO § 462.1 What is the scope of this part?
Regulatory Flexibility Act Certification Access at: http://www.gpoaccess.gov/nara/ The regulations in this part establish
(pages 61582 and 61583), and index.html. the—
Paperwork Reduction Act of 1995 (page (Catalog of Federal Domestic Assistance (a) Procedures the Secretary uses to
61583). Number does not apply.) determine the suitability of
Paperwork Reduction Act of 1995 List of Subjects in 34 CFR Part 462 standardized tests for use in the
National Reporting System for Adult
The Paperwork Reduction Act of 1995 Administrative practice, Adult Education (NRS) to measure educational
does not require you to respond to a education, Grants program—education, gain of participants in an adult
collection of information unless it Incorporation by reference, and education program required to report
displays a valid OMB control number. Reporting and recordkeeping under the NRS; and
With the exception of §§ 462.10 through requirements. (b) Procedures States and local
462.14, we display the valid OMB Dated: January 7, 2008. eligible providers must follow when
control number assigned to the Margaret Spellings, measuring educational gain for use in
collection of information in these final the NRS.
Secretary of Education.
regulations at the end of the affected
sections of the regulations. ■ For the reasons discussed in the (Authority: 20 U.S.C. 9212)
preamble, the Secretary amends title 34
Intergovernmental Review § 462.2 What regulations apply?
of the Code of Federal Regulations by
These regulations are subject to adding a new part 462 to read as The following regulations apply to
Executive Order 12372 and the follows: this part:
regulations in 34 CFR part 79. One of (a) The Education Department General
the objectives of the Executive order is PART 462—MEASURING Administrative Regulations (EDGAR) as
to foster an intergovernmental EDUCATIONAL GAIN IN THE follows:
partnership and a strengthened NATIONAL REPORTING SYSTEM FOR (1) 34 CFR part 74 (Administration of
federalism by relying on processes ADULT EDUCATION Grants and Agreements with Institutions
developed by State and local of Higher Education, Hospitals, and
Subpart A—General Other Non-Profit Organizations).
governments for coordination and
Sec. (2) 34 CFR part 76 (State-
review of proposed Federal financial 462.1 What is the scope of this part?
assistance. Administered Programs).
462.2 What regulations apply? (3) 34 CFR part 77 (Definitions that
In accordance with the order, we 462.3 What definitions apply?
intend this document to provide early Apply to Department Regulations).
462.4 What are the transition rules for using
notification of the Department’s specific tests to measure educational gain for the
(4) 34 CFR part 79 (Intergovernmental
plans and actions for this program. National Reporting System for Adult Review of Department of Education
Education (NRS)? Programs and Activities).
Assessment of Educational Impact (5) 34 CFR part 80 (Uniform
Subpart B—What Process Does the Administrative Requirements for Grants
In the NPRM we requested comments Secretary Use To Review the Suitability of
on whether the proposed regulations Tests for Use in the NRS?
and Cooperative Agreements to State
would require transmission of and Local Governments).
462.10 How does the Secretary review (6) 34 CFR part 81 (General Education
information that any other agency or tests?
authority of the United States gathers or 462.11 What must an application contain?
Provisions Act—Enforcement).
makes available. 462.12 What procedures does the Secretary (7) 34 CFR part 82 (New Restrictions
Based on the response to the NPRM use to review the suitability of tests? on Lobbying).
and on our review, we have determined 462.13 What criteria and requirements does (8) 34 CFR part 84 (Governmentwide
that these final regulations do not the Secretary use for determining the Requirements for Drug-Free Workplace
require transmission of information that suitability of tests? (Financial Assistance)).
any other agency or authority of the 462.14 How often and under what (9) 34 CFR part 85 (Governmentwide
circumstances must a test be reviewed by Debarment and Suspension
United States gathers or makes
the Secretary? (Nonprocurement)).
available.
Subpart C—[Reserved] (10) 34 CFR part 86 (Drug and Alcohol
Electronic Access to This Document Abuse Prevention).
Subpart D—What Requirements Must
You can view this document, as well States and Local Eligible Providers Follow (11) 34 CFR part 97 (Protection of
as all other Department of Education When Measuring Educational Gain? Human Subjects).
documents published in the Federal 462.40 Must a State have an assessment

(12) 34 CFR part 98 (Student Rights in
Register, in text or Adobe Portable policy? Research, Experimental Programs, and
Document Format (PDF) on the Internet 462.41 How must tests be administered in Testing).
at the following site: http://www.ed.gov/ order to accurately measure educational (13) 34 CFR part 99 (Family
news/fedregister. gain? Educational Rights and Privacy).
(b) The regulations in this part 462. the National Reporting System for Adult Subpart B—What Process Does the
(Authority: 20 U.S.C. 9212) Education (also known as NRS Secretary Use To Review the
Implementation Guidelines) posted on Suitability of Tests for Use in the NRS?
§ 462.3 What definitions apply? the Internet at: http://www.nrsweb.org.
(a) Definitions in the Adult Education A copy of the Guidelines is also § 462.10 How does the Secretary review
available from the U.S. Department of tests?
and Family Literacy Act (Act). The
following terms used in these Education, Division of Adult Education (a) The Secretary only reviews tests
regulations are defined in section 203 of and Literacy, 400 Maryland Avenue, under this part that are submitted by a
the Adult Education and Family SW., room 11159, Potomac Center Plaza, test publisher.
Literacy Act, 20 U.S.C. 9202 (Act): Washington, DC 20202–7240. (b) A test publisher that wishes to
Adult education, Local eligible provider means an have the suitability of its test
Eligible provider, ‘‘eligible provider’’ as defined in the Act determined by the Secretary under this
Individual of limited English that operates an adult education part must submit an application to the
proficiency, program that is required to report under Secretary, in the manner the Secretary
Individual with a disability, the NRS. may prescribe, by April 14, 2008, and,
Literacy. State means ‘‘State’’ and ‘‘Outlying thereafter, by October 1 of each year.
(b) Other definitions. The following area’’ as defined in the Act. (Authority: 20 U.S.C. 9212)
definitions also apply to this part: Test means a standardized test,
Adult basic education (ABE) means § 462.11 What must an application
assessment, or instrument that has a contain?
instruction designed for an adult whose formal protocol on how it is to be
educational functioning level is administered. These protocols include, (a) Application content and format. In
equivalent to a particular ABE literacy for example, the use of parallel, equated order for the Secretary to determine
level listed in the NRS educational forms, testing conditions, time allowed whether a standardized test is suitable
functioning level table in § 462.44. for the test, standardized scoring, and for measuring the gains of participants
Adult education population means the amount of instructional time a in an adult education program required
individuals— student needs before post-testing. to report under the NRS, a test publisher
(1) Who are 16 years of age or older; Violation of these protocols often must—
(2) Who are not enrolled or required (1) Include with its application
invalidates the test scores. Tests are not
to be enrolled in secondary school information listed in paragraphs (b)
limited to traditional paper and pencil
under State law; and through (i) of this section, and, if
(or computer-administered) instruments
(3) Who— applicable, the information listed in
for which forms are constructed prior to
(i) Lack sufficient mastery of basic paragraph (j) of this section;
administration to examinees. Tests may
educational skills to enable the (2) Provide evidence that it holds a
also include adaptive tests that use
individuals to function effectively in registered copyright of a test or is
computerized algorithms for selecting
society; licensed by the copyright holder to sell
(ii) Do not have a secondary school and administering items in real time;
or distribute a test.
diploma or its recognized equivalent, however, for such instruments, the size (3)(i) Arrange the information in its
and have not achieved an equivalent of the item pool and the method of item application in the order it is presented
level of education; or selection must ensure negligible overlap in paragraphs (b) through (j) of this
(iii) Are unable to speak, read, or in items across pre- and post-testing. section; or
write the English language. Test administrator means an (ii) Include a table of contents in its
Adult secondary education (ASE) individual who is trained to administer application that identifies the location
means instruction designed for an adult tests the Secretary determines to be of the information required in
whose educational functioning level is suitable under this part. paragraphs (b) through (j) of this section.
equivalent to a particular ASE literacy Test publisher means an entity, (4) Submit to the Secretary three
level listed in the NRS educational individual, organization, or agency that copies of its application.
functioning level table in § 462.44. owns a registered copyright of a test or (b) General information. (1) A
Content domains, content is licensed by the copyright holder to statement, in the technical manual for
specifications, or NRS skill areas mean, sell or distribute a test. the test, of the intended purpose of the
for the purpose of the NRS, reading, (Authority: 20 U.S.C. 9202, 9212) test and how the test will allow
writing, and speaking the English examinees to demonstrate the skills that
language, numeracy, problem solving, § 462.4 What are the transition rules for are associated with the NRS educational
using tests to measure educational gain for
English language acquisition, and other the National Reporting System for Adult
functioning levels in § 462.44.
literacy skills as defined by the Education (NRS)? (2) The name, address, e-mail address,
Secretary. and telephone and fax numbers of a
Educational functioning levels mean A State or a local eligible provider contact person to whom the Secretary
the ABE, ASE, and ESL literacy levels, may continue to measure educational may address inquiries.
as provided in § 462.44, that describe a gain for the NRS using a test that was (3) A summary of the precise editions,
set of skills and competencies that identified in the Guidelines until the forms, levels, and, if applicable, sub-
students demonstrate in the NRS skill Secretary announces through a notice tests and abbreviated tests that the test
areas. published in the Federal Register a publisher is requesting that the
English-as-a-second language (ESL) deadline by which States and local Secretary review and determine to be
means instruction designed for an adult eligible providers must use only tests suitable for use in the NRS.
whose educational functioning level is that the Secretary has reviewed and (c) Development. Documentation of
equivalent to a particular ESL literacy determined to be suitable for use in the how the test was developed, including
level listed in the NRS educational NRS under this part. a description of—
functioning level table in § 462.44. (Approved by the Office of Management and (1) The nature of samples of
Guidelines means the Implementation Budget under control number 1830–0027) examinees administered the test during
Guidelines: Measures and Methods for (Authority: 20 U.S.C. 9212) pilot or field testing, such as—
(i) The number of examinees the items or tasks to the NRS measuring educational gain for the NRS,
administered each item; educational functioning levels and their i.e., documentation that the test
(ii) How similar the sample or qualifications for doing so, particularly measures what it is intended to
samples of examinees used to develop their familiarity with adult education measure, including—
and evaluate the test were to the adult and the NRS educational functioning (1) The extent to which the raw or
education population of interest to the levels; and scale scores and the educational
NRS; and (6) The extent to which the judgments functioning classifications associated
(iii) The steps, if any, taken to ensure of the subject matter experts agree. with the test correlate (or agree) with
that the examinees were motivated (f) Match of scores to NRS educational scores or classifications associated with
while responding to the test; and functioning levels. Documentation of the other tests designed or intended to
(2) The steps taken to ensure the adequacy of the procedure used to assess educational gain in the same
quality of test items or tasks, such as— translate the performance of an adult education population as the NRS;
(i) The extent to which items or tasks examinee on a particular test to an (2) The extent to which the raw or
on the test were reviewed for fairness estimate of the examinee’s standing scale scores are related to other relevant
and sensitivity; and with respect to the NRS educational variables, such as teacher evaluation,
(ii) The extent to which items or tasks functioning levels in § 462.44, hours of instruction, or other measures
on the test were screened for the including— that may be related to test performance;
adequacy of their psychometric (1) The standard-setting procedures (3) The adequacy of the research
properties. used to establish cut scores for designs associated with these sources of
(3) The procedures used to assign transforming raw or scale scores on the evidence (see paragraph (g)(3) of this
items to— test into estimates of an examinee’s NRS section); and
(i) Forms, for tests that are educational functioning level; (4) Other evidence demonstrating that
constructed prior to being administered (2) If judgment-based procedures were the test measures gains in educational
to examinees; or used— functioning resulting from adult
(ii) Examinees, for adaptive tests in (i) The number of subject-matter education and not from other construct-
which items are selected in real time. experts who provided judgments, and irrelevant variables, such as practice
(d) Maintenance. Documentation of their qualifications; and effects.
how the test is maintained, including a (ii) Evidence of the extent to which (i) Other information. (1) A
description of— the judgments of subject-matter experts description of the manner in which test
(1) How frequently, if ever, new forms agree; administration time was determined,
of the test are developed; (3) The standard error of each cut and an analysis of the speededness of
(2) The steps taken to ensure the score, and how it was established; and the test.
comparability of scores across forms of (4) The extent to which the cut scores (2) Additional guidance on the
the test; might be expected to differ if they had interpretation of scores resulting from
(3) The steps taken to maintain the been established by a different (though any modifications of the tests for an
security of the test; similar) panel of experts. individual with a disability.
(4) A history of the test’s use, (g) Reliability. Documentation of the (3) The manual provided to test
including the number of times the test degree of consistency in performance administrators containing procedures
has been administered; and across different forms of the test in the and instructions for test security and
(5) For a computerized adaptive test, absence of any external interventions, administration.
the procedures used to— including— (4) A description of the training or
(i) Select subsets of items for (1) The correlation between raw (or certification required of test
administration; scale) scores across alternate forms of administrators and scorers by the test
(ii) Determine the starting point and the test or, in the case of computerized publisher.
termination conditions; adaptive tests, across alternate (5) A description of retesting (e.g., re-
(iii) Score the test; and administrations of the test; administration of a test because of
(iv) Control for item exposure. (2) The consistency with which problems in the original administration
(e) Match of content to the NRS examinees are classified into the same such as the test taker becomes ill during
educational functioning levels (content NRS educational functioning levels the test and cannot finish, there are
validity). Documentation of the extent to across forms of the test. Information external interruptions during testing, or
which the items or tasks on the test regarding classification consistency there are administration errors)
cover the skills in the NRS educational should be reported for each NRS procedures and the analysis upon which
functioning levels in § 462.44, educational functioning level that the the criteria for retesting are based.
including— test is being considered for use in (6) Such other evidence as the
(1) Whether the items or tasks on the measuring; Secretary may determine is necessary to
test require the types and levels of skills (3) The adequacy of the research establish the test’s compliance with the
used to describe the NRS educational design leading to the estimates of the criteria and requirements the Secretary
functioning levels; reliability of the test, including— uses to determine the suitability of tests
(2) Whether the items or tasks (i) The size of the sample(s); as provided in § 462.13.
measure skills that are not associated (ii) The similarity between the (j) Previous tests. (1) For a test used
with the NRS educational functioning sample(s) used in the data collection to measure educational gain in the NRS
levels; and the adult education population; and before the effective date of these
(3) Whether aspects of a particular (iii) The steps taken to ensure the regulations that is submitted to the
NRS educational functioning level are motivation of the examinees; and Secretary for review under this part, the
not covered by any of the items or tasks; (4) Any other information explaining test publisher must provide
(4) The procedures used to establish the methodology and procedures used documentation of periodic review of the
the content validity of the test; to measure the reliability of the test. content and specifications of the test to
(5) The number of subject-matter (h) Construct validity. Documentation ensure that the test continues to reflect
experts who provided judgments linking of the appropriateness of a given test for NRS educational functioning levels.
(2) For a test first published five years size of the item pool and the method of Secretary’s reasons for determining that
or more before the date it is submitted item selection must ensure negligible the test was unsuitable.
to the Secretary for review under this overlap in items across pre- and post- (3) The Secretary reviews the
part, the test publisher must provide testing; additional information submitted by the
documentation of periodic review of the (iv) Includes a test that samples one test publisher and makes a final
content and specifications of the test to or more of the major content domains of determination regarding the suitability
ensure that the test continues to reflect the NRS educational functioning levels of the test for use in the NRS.
NRS educational functioning levels. of ABE, ESL, or ASE with sufficient (i) If the Secretary’s decision is
(3) For a test that has not changed in numbers of questions to represent unchanged and the test remains
the seven years since the Secretary adequately the domain or domains; and unsuitable for use in the NRS, the
determined, under § 462.13, that it was (v) Includes the information Secretary notifies the test publisher, and
suitable for use in the NRS that is again prescribed by the Secretary, including this action concludes the review
being submitted to the Secretary for the information in § 462.11 of this part. process.
review under this part, the test (b) Secretary’s determination. (1) The (ii) If the Secretary’s decision changes
publisher must provide updated data Secretary determines whether a test and the test is determined to be suitable
supporting the validity of the test for meets the criteria and requirements in for use in the NRS, the Secretary follows
use in classifying adult learners with § 462.13 after taking into account the the procedures in paragraph (c) of this
respect to the NRS educational advice of the experts described in section.
functioning levels and the measurement (e) Revocation. (1) The Secretary’s
paragraph (a)(1) of this section.
of educational gain as defined in determination regarding the suitability
(2) For tests that contain multiple sub-
§ 462.43 of this part. of a test may be revoked if the Secretary
tests measuring content domains other
(4) If a test has been substantially determines that—
than those of the NRS educational (i) The information the publisher
revised—for example by changing its functioning levels, the Secretary
structure, number of items, content submitted as a basis for the Secretary’s
determines the suitability of only those review of the test was inaccurate; or
specifications, item types, or sub-tests— sub-tests covering the domains of the
from the most recent edition reviewed (ii) A test has been substantially
NRS educational functioning levels. revised—for example, by changing its
by the Secretary under this part, the test (c) Suitable tests. If the Secretary
publisher must provide an analysis of structure, number of items, content
determines that a test satisfies the specifications, item types, or sub-tests.
the revisions, including the reasons for criteria and requirements in § 462.13 (2) The Secretary notifies the test
the revisions, the implications of the and, therefore, is suitable for use in the publisher of the—
revisions for the comparability of scores NRS, the Secretary— (i) Secretary’s decision to revoke the
on the current test to scores on the (1) Notifies the test publisher of the determination that the test is suitable for
previous test, and results from validity, Secretary’s decision; and use in the NRS; and
reliability, and equating or standard- (2) Annually publishes in the Federal (ii) Reasons for the Secretary’s
setting studies undertaken subsequent Register and posts on the Internet at revocation.
to the revisions. http://www.nrsweb.org a list of the (3) Within 30 days after the Secretary
(Authority: 20 U.S.C. 9212) names of tests and the educational notifies a test publisher of the decision
functioning levels the tests are suitable to revoke a determination that a test is
§ 462.12 What procedures does the to measure in the NRS. A copy of the suitable for use in the NRS, the test
Secretary use to review the suitability of
list is also available from the U.S. publisher may request that the Secretary
tests?
Department of Education, Office of reconsider the decision. This request
(a) Review. (1) When the Secretary must be accompanied by documentation
Vocational and Adult Education,
receives a complete application from a and information that address the
Division of Adult Education and
test publisher, the Secretary selects Secretary’s reasons for revoking the
Literacy, 400 Maryland Avenue, SW.,
experts in the field of educational determination that the test is suitable for
room 11159, Potomac Center Plaza,
testing and assessment who possess use in the NRS.
Washington, DC 20202–7240.
appropriate advanced degrees and (4) The Secretary reviews the
(d) Unsuitable tests. (1) If the
experience in test development or information submitted by the test
Secretary determines that a test does not
psychometric research, or both, to publisher and makes a final
satisfy the criteria and requirements in
advise the Secretary on the extent to determination regarding the suitability
§ 462.13 and, therefore, is not suitable
which a test meets the criteria and of the test for use in the NRS.
for use in the NRS, the Secretary notifies
requirements in § 462.13. (5) If the Secretary revokes the
(2) The Secretary reviews and the test publisher of the Secretary’s
decision and of the reasons why the test determination regarding the suitability
determines the suitability of a test only of a test, the Secretary publishes in the
if an application— does not meet those criteria and
requirements. Federal Register and posts on the
(i) Is submitted by a test publisher; Internet at http://www.nrsweb.org a
(ii) Meets the deadline established by (2) Within 30 days after the Secretary
notifies a test publisher that its test is notice of that revocation along with the
the Secretary;
(iii) Includes a test that— not suitable for use in the NRS, the test date by which States and local eligible
(A) Has two or more secure, parallel, publisher may request that the Secretary providers must stop using the revoked
equated forms of the same test—either reconsider the Secretary’s decision. This test. A copy of the notice of revocation
traditional paper and pencil or request must be accompanied by— is also available from the U.S.
computer-administered instruments— (i) An analysis of why the information Department of Education, Office of
for which forms are constructed prior to and documentation submitted meet the Vocational and Adult Education,
administration to examinees; or criteria and requirements in § 462.13, Division of Adult Education and
(B) Is an adaptive test that uses notwithstanding the Secretary’s earlier Literacy, 400 Maryland Avenue, SW.,
computerized algorithms for selecting decision to the contrary; and room 11159, Potomac Center Plaza,
and administering items in real time; (ii) Any additional documentation Washington, DC 20202–7240.
however, for such an instrument, the and information that address the (Authority: 20 U.S.C. 9212)
§ 462.13 What criteria and requirements and administering items in real time; Subpart C—[Reserved]
does the Secretary use for determining the however, for such an instrument, the
suitability of tests? size of the item pool and the method of Subpart D—What Requirements Must
In order for the Secretary to consider item selection must ensure negligible States and Local Eligible Providers
a test suitable for use in the NRS, the overlap in items across pre- and post- Follow When Measuring Educational
test or the test publisher, if applicable, testing. Scores associated with these Gain?
must meet the following criteria and alternate administrations must be
requirements: equivalent in meaning. § 462.40 Must a State have an assessment
(a) The test must measure the NRS (f) For a test that has been modified policy?
educational functioning levels of for individuals with disabilities, the test (a) A State must have a written
members of the adult education publisher must— assessment policy that its local eligible
population. (1) Provide documentation that it providers must follow in measuring
(b) The test must sample one or more followed the guidelines provided in the educational gain and reporting data in
of the major content domains of the NRS Testing Individuals With Disabilities the NRS.
educational functioning levels of ABE, section of the 1999 edition of the (b) A State must submit its assessment
ESL, or ASE with sufficient numbers of Standards for Educational and policy to the Secretary for review and
questions to adequately represent the Psychological Testing; approval at the time it submits its
domain or domains. (2) Provide documentation of the annual statistical report for the NRS.
(c)(1) The test must meet all appropriateness and feasibility of the (c) The State’s assessment policy
applicable and feasible standards for modifications relevant to test must—
test construction and validity provided performance; and (1) Include a statement requiring that
in the 1999 edition of the Standards for (3)(i) Recommend educational local eligible providers measure the
Educational and Psychological Testing, functioning levels based on the educational gain of all students who
prepared by the Joint Committee on information obtained from adult receive 12 hours or more of instruction
Standards for Educational and education students who participated in in the State’s adult education program
Psychological Testing of the American the pilot or field test and who have the with a test that the Secretary has
Educational Research Association, the disability for which the test has been determined is suitable for use in the
American Psychological Association, modified; and NRS;
and the National Council on (ii) Provide documentation of the (2) Identify the pre- and post-tests that
Measurement in Education incorporated adequacy of the procedures used to the State requires local eligible
by reference in this section. The translate the performance of adult providers to use to measure the
Director of the Federal Register education students with the disability educational gain of ABE, ESL, and ASE
approves this incorporation by reference for whom the test has been modified to students;
in accordance with 5 U.S.C. 552(a) and an estimate of the examinees’ standing (3)(i) Indicate when, in calendar days
1 CFR part 51. You may obtain a copy with respect to the NRS educational or instructional hours, local eligible
from the American Psychological functioning levels. providers must administer pre- and
Association, Inc., 750 First Street, NE., (Authority: 20 U.S.C. 9212) post-tests to students; and
Washington, DC 20002. You may (ii) Ensure that the time for
§ 462.14 How often and under what administering the post-test is long
inspect a copy at the Department of
circumstances must a test be reviewed by enough after the pre-test to allow the
Education, room 11159, 550 12th Street, the Secretary?
SW., Washington, DC 20202 or at the test to measure educational gains
(a) The Secretary’s determination that according to the test publisher’s
National Archives and Records
a test is suitable for use in the NRS is guidelines;
Administration (NARA). For
in effect for a period of seven years from (4) Specify the score ranges tied to
information on the availability of this
the date of the Secretary’s written educational functioning levels for
material at NARA, call (202) 741–6030, notification to the test publisher, unless
or go to: http://www.archives.gov/ placement and for reporting gains for
otherwise indicated by the Secretary. accountability;
federal_register/ After that time, if the test publisher
code_of_federal_regulations/ (5) Identify the skill areas the State
wants the test to be used in the NRS, the intends to require local eligible
ibr_locations.html. test must be reviewed again by the
(2) If requested by the Secretary, a test providers to assess in order to measure
Secretary so that the Secretary can educational gain;
publisher must explain why it believes
determine whether the test continues to (6) Include the guidance the State
that certain standards in the 1999
be suitable for use in the NRS. provides to local eligible providers on
edition of the Standards for Educational (b) If a test that the Secretary has
and Psychological Testing were not testing and placement of an individual
determined is suitable for use in the with a disability or an individual who
applicable or were not feasible to meet. NRS is substantially revised—for
(d) The test must contain the is unable to be tested because of a
example, by changing its structure, disability;
publisher’s guidelines for retesting,
number of items, content specifications, (7) Describe the training requirements
including time between test-taking,
item types, or sub-tests—and the test that staff must meet in order to be
which are accompanied by appropriate
publisher wants the test to continue to qualified to administer and score each
justification.
(e) The test must— be used in the NRS, the test publisher test selected by the State to measure the
(1) Have two or more secure, parallel, must submit, as provided in educational gains of students;
equated forms of the same test—either § 462.11(j)(4), the substantially revised (8) Identify the alternate form or forms
traditional paper and pencil or test or version of the test to the of each test that local eligible providers
computer administered instruments— Secretary for review so that the must use for post-testing;
for which forms are constructed prior to Secretary can determine whether the (9) Indicate whether local eligible
administration to examinees; or test continues to be suitable for use in providers must use a locator test for
(2) Be an adaptive test that uses the NRS. guidance on identifying the appropriate
computerized algorithms for selecting (Authority: 20 U.S.C. 9212) pre-test;
(10) Describe the State’s policy for the instruments—for which forms are (2) If a State’s assessment policy
initial placement of a student at each constructed prior to administration to requires a local eligible provider to test
NRS educational functioning level using examinees to pre-test and determine the a student in multiple skill areas, but the
test scores; initial placement of students; or student will receive instruction in fewer
(11) Describe the State’s policy for (ii) Administer post-tests with an than all of the skill areas, the local
using the post-test for measuring adaptive test that uses computerized eligible provider must place the student
educational gain and for advancing algorithms for selecting and in an educational functioning level that
students across educational functioning administering items in real time; is equivalent to the student’s lowest test
levels; however, for such an instrument, the score for any of the skill areas—
(12) Describe the pre-service and in- size of the item pool and the method of (i) Tested under § 462.41(b) and (c);
service staff training that the State or item selection must ensure negligible and
local eligible providers will provide, overlap in items across pre- and post- (ii) In which the student will receive
including training— testing; and instruction.
(i) For staff who either administer or (4) Administer post-tests to students (Approved by the Office of Management and
score each of the tests used to measure in the same skill areas as the pre-test. Budget under control number 1830–0027)
educational gain; (d) Other requirements. (1) A local (Authority: 20 U.S.C. 9212)
(ii) For teachers and other local staff eligible provider must administer a test
involved in gathering, analyzing, using only staff who have been trained § 462.43 How is educational gain
compiling, and reporting data for the to administer the test. measured?
NRS; and (2) A local eligible provider may use
(iii) That includes the following (a)(1) Educational gain is measured by
the results of a test in the NRS only if
topics: comparing the student’s initial
the test was administered in a manner
(A) NRS policy, accountability educational functioning level, as
that is consistent with the State’s
policies, and the data collection process. measured by the pre-test described in
assessment policy and the test
(B) Definitions of measures. § 462.41(b), with the student’s
publisher’s guidelines.
(C) Conducting assessments; and educational functioning level as
(13) Identify the State or local agency (Approved by the Office of Management and measured by the post-test described in
responsible for providing pre- and in- Budget under control number 1830–0027) § 462.41(c).
service training. (Authority: 20 U.S.C. 9212)
Example: A State’s assessment policy
(Approved by the Office of Management and § 462.42 How are tests used to place requires its local eligible providers to test
Budget under control number 1830–0027) students at an NRS educational functioning students in reading and numeracy. The
(Authority: 20 U.S.C. 9212) level? student scores lower in reading than in
numeracy. As described in § 462.42(d)(1), the
(a) A local eligible provider must use local eligible provider would use the
§ 462.41 How must tests be administered
in order to accurately measure educational the results of the pre-test described in student’s reading score to place the student
gain? § 462.41(b) to initially place students at in an educational functioning level. To
the appropriate NRS educational measure educational gain, the local eligible
(a) General. A local eligible provider
functioning level. provider would compare the reading score on
must measure the educational gains of the pre-test with the reading score on the
students using only tests that the (b) A local eligible provider must use
the results of the post-test described in post-test.
Secretary has determined are suitable
for use in the NRS and that the State has § 462.41(c)— (2) A student is considered to have
identified in its assessment policy. (1) To determine whether students made an educational gain when the
(b) Pre-test. A local eligible provider have completed one or more student’s post-test indicates that the
must— educational functioning levels or are student has completed one or more
(1) Administer a pre-test to measure a progressing within the same level; and educational functioning levels above the
student’s educational functioning level (2) To place students at the level in which the student was placed
at intake, or as soon as possible appropriate NRS educational by the pre-test.
thereafter; functioning level. (b) If a student is not post-tested, then
(2) Administer the pre-test to students (c)(1) States and local eligible no educational gain can be measured for
at a uniform time, according to its providers are not required to use all of that student and the local eligible
State’s assessment policy; and the skill areas described in the NRS provider must report the student in the
(3) Administer pre-tests to students in educational functioning levels to place same educational functioning level as
the skill areas identified in its State’s students. initially placed for NRS reporting
assessment policy. (2) States and local eligible providers purposes.
(c) Post-test. A local eligible provider must test and report on the skill areas
(Approved by the Office of Management and
must— most relevant to the students’ needs and Budget under control number 1830–0027)
(1) Administer a post-test to measure to the programs’ curriculum.
(Authority: 20 U.S.C. 9212)
a student’s educational functioning (d)(1) If a State’s assessment policy
level after a set time period or number requires a local eligible provider to test § 462.44 Which educational functioning
of instructional hours; a student in multiple skill areas and the levels must States and local eligible
(2) Administer the post-test to student will receive instruction in all of providers use to measure and report
students at a uniform time, according to the skill areas, the local eligible educational gain in the NRS?
its State’s assessment policy; provider must place the student in an States and local eligible providers
(3)(i) Administer post-tests with a educational functioning level that is must use the NRS educational
secure, parallel, equated form of the equivalent to the student’s lowest test functioning levels in the following
same test—either traditional paper and score for any of the skill areas tested functioning level table:
pencil or computer-administered under § 462.41(b) and (c). BILLING CODE 4001–01–P
ER14ja08.000</GPH>
ER14ja08.001</GPH>
ER14ja08.002</GPH>
(Approved by the Office of Management and (Authority: 20 U.S.C. 9212)

Budget under control number 1830–0027) [FR Doc. 08–69 Filed 1–11–08; 8:45 am]
BILLING CODE 4001–01–C

ER14ja08.003</GPH>

Rule: Measuring Educational Gain in The National Reporting System For Adult Education

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Rule: Measuring Educational Gain in The National Reporting System For Adult Education

Uploaded by

Copyright:

Available Formats

Monday,

January 14, 2008

Content of an application—Involvement responding to the test. The regulations Comments: A commenter

documents published in the Federal 462.40 Must a State have an assessment

(Approved by the Office of Management and (Authority: 20 U.S.C. 9212)

BILLING CODE 4001–01–C

You might also like