Subject-Area Knowledge Measured by Scores on the National

Association of State Boards of Geology (ASBOG) Fundamentals

Examination and the Implications for Academic Preparation
John W. Williams
Jack L. Warner
Steven P. Warner

Department of Geology, San Jos State University, San Jos, CA 95192-0102

TEST, Inc., Aurora, CO, 80014-5369
TEST, Inc. Aurora, CO 80014-5369

ABSTRACT
The National Association of State Boards of Geology
(ASBOG) has administered approximately fifty-one
hundred examinations for the licensing of geologists
since the initial examination offering in 1992. The
examination consists of a fundamentals portion and a
practice portion. Each portion is 4 hours in length. The
average passing rate across all administrations of the
fundamentals examination is 58% and is 68% for the
practice examination.
For each examination, the subject matter tested is
divided into nine areas or subject area domains based
upon the results of the task analysis (survey of the
practicing profession) that guides the examination
blueprint. For the fundamentals examination, the
distribution of questions among domains is: field
methods and remote sensing (28% of 110 examination
questions), mineralogy, petrology, petrography, and
geochemistry (14%), sedimentology, stratigraphy, and
paleontology (10%), geomorphology (6%), structural
geology and tectonics (9%), geophysics and seismology
(4%), hydrogeology (25%), engineering geology (3%),
and mineral, petroleum, and energy resources (1%).
The candidate success in each domain on the
fundamentals examinations was determined for the last
five administrations of the examinations during the
period 2002-2003. The average percentage of questions
answered correctly in each domain was: field methods
and remote sensing (67% of questions answered
correctly), mineralogy, petrology, petrography, and
geochemistry (58%), sedimentology, stratigraphy, and
paleontology (56%), geomorphology (63%), structural
geology and tectonics (64%), geophysics and seismology
(60%), hydrogeology (67%), engineering geology in
combination with mineral, petroleum, and energy
resources (67%).
Candidates are doing poorer in those subject areas
traditionally believed to be the fundamental subject
areas of an undergraduate geology education
(mineralogy, petrology, sedimentology, etc).

INTRODUCTION
The National Association of State Boards of Geology
(ASBOG) has administered more than five thousand
examinations for the licensing of geologists since 1992
(National Association of State Boards of Geology, 2002).
Twenty-seven of the 31 states requiring licensure of
geologists use the two-part multiple-choice ASBOG
examination. This examination consists of two parts: 1) a
fundamentals portion designed for recent geology
graduates with bachelors degrees and 2) a practice
portion for those with the undergraduate degree and the
state-mandated period of professional experience. The
format of the exam is exclusively multiple choice. This
format permits more accurate statistical evaluation of
374

results and comparison of candidate performance

between exam administrations than open-ended or essay
questions, which are open to subjective interpretation by
reviewers (Ebel and Frisbie, 1991). The distribution of
questions is based upon a review of the practicing
profession (Figure 1). This paper focuses on the
Fundamentals of Geology (FG) portion of the
examination and implications that can be drawn for the
design of academic curricula.

CANDIDATE PERFORMANCE
For the period 2002-2003, the average percentage of
questions answered correctly in each domain (subject
area) was: field methods and remote sensing (67%),
mineralogy, petrology, petrography and geochemistry
(58%), sedimentology, stratigraphy and paleontology
(55%), geomorphology (62%), structural geology and
tectonics (62%), geophysics and seismology (58%),
hydrogeology
(66%),
engineering
geology
in
combination with mineral, petroleum and energy
resources (66%) (Figure 2).
On average, candidates are performing less well in
subject areas typically considered the core subject areas
of an undergraduate education (mineralogy, petrology,
sedimentology, etc). Some variability in the performance
of candidates in the various subject matter domains has
occurred over the last six administrations of the
examinations (Figure 2) (Williams et al., 2002).
On the fundamentals examinations, about 60% of
those students who have graduated with degrees in
geology are successful. The colleges and universities
have stated that each graduated student has met the
schools and departments requirements for the degree in
geology with an overall C average. With few
exceptions, 100 percent of the candidates taking the
examinations have earned degrees, but only 60 percent
can pass the test. Why? Certainly one can blame the test
for many imagined and possibly real reasons including:
inappropriate construction as to content, or deliberate
design to fail candidates thereby maintaining and
financially favoring the established pool of members.
Although it is convenient to blame the exam, an
approach near and dear to the hearts of all college
students, that is probably not appropriate.
Candidates fail examinations for a variety of reasons,
but generally, failures fall into one or two categories
including: a) the candidate did not know what to expect
on the exam; b) the candidate was not academically
prepared in the content knowledge addressed by the
exam: or c) the exam was not appropriately designed. In
the case of the ASBOG examinations, through the
Candidate Handbook, the candidate is provided with
significant amounts of information including numerous
example questions to help prepare them for the type and
scope of questions they will encounter. This handbook is
most valuable to those anticipating taking the
examination and is available on-line at the ASBOG

Journal of Geoscience Education, v. 52, n. 4, September, 2004, p. 374-378

Content Domains

No.

FG %

Content Domains

A. Field Methods & Remote Sensing

B. Mineralogy, Petrology, Petrography,
& Geochemistry
C. Sedimentology, Stratigraphy, &
Paleontology
D. Geomorphology
E. Structural Geology & Tectonics
F. Geophysics & Seismology
G. Hydrogeology
H. Engineering Geology
I. Mineral, Petroleum, & Energy
Resources
TOTALS

32

29.1

15

13.6

11

10.0

7
10
4
27
3

6.4
9.1
3.6
24.5
2.7

0.9

110

100.1

A. Field Methods & Remote Sensing

(n = 32)
HI. Engineering Geology/Mineral,
Petroleum, & Energy Resources
(n = 4)
G. Hydrogeology (n = 27)
E. Structural Geology & Tectonics (n
= 10)
D. Geomorphology (n = 7)
B. Mineralogy, Petrology,
Petrography, & Geochemistry (n
= 15)
F. Geophysics & Seismology (n = 4)
C. Sedimentology, Stratigraphy, &
Paleontology (n = 11)

Avg.

Range1

66.8

63-71

66.3

52-82

65.8

59-71

62.0

54-66

61.8

55-69

58.0

54-66

57.5

47-74

55.0

50-62

Figure 1. National Association of State Boards of

Geology Fundamentals of Geology test blueprint Figure 2. Fundamentals of geology examination
number and percent of questions of each content (September 2000 March 2003) average percent and
range1 correct for each content domain (sorted by
domain.
percent correct). 1Range in percent correct for
examinations given September 2000 March 2003

website for anyone interested in the examination. It

contains information about exam development, passing
scores, distribution of questions among subject areas,
and representative questions. These questions (and the
answers) are provided to illustrate the type and the
degree of difficulty of questions challenging the
applicants (ASBOG Web Site, 2004).
Additional factors should be considered as
influencing examination results and candidate
performance. Academic programs are being modified to
attract and keep more students during these times of
significant declines in undergraduate enrollments
(American Geological Institute, 2003). One way to do this
is to make the course requirements less onerous, less
rigorous. Programs are requiring fewer semesters of
calculus and permit students to substitute non-calculus
based physics for the traditional calculus-based physics
classes. Course content needs to be evaluated. Is the
material being presented in the course what the student
needs to be a competent geologist? This question raises
the old unresolved debate of what is the role of the
college and university in educating a student educate
(or as viewed by some, train them) to get and do a good
job in the profession or to educate them to advance the
science through research, etc. Thus, the attitude of the
faculty member presenting the course has a great deal of
influence on the material included and emphasized. Is
the bulk of the material applied or theoretical?

COMPARISON OF CANDIDATE
PERFORMANCE AMONG DISCIPLINES

sample size,
length of time that has elapsed between examination
and reporting of data ,
format of the examination (multiple choice, essay,
etc.), and
structure of the examination (specialty, general, etc.).

Long-term data sets are not easily obtained in

part because some licensure examinations have only
been given for relatively short periods. To date, few
disciplines have reviewed and presented the candidates
performance on the subject matter domains within the
discipline. A review of information that is currently
available from licensing groups indicates that the range
in passing is wide. For example, the range in passing
percentage within the various sub-disciplines of the
engineering profession varies from 28 to 85 percent
(National Council of Examiners for Engineering and
Surveying, 2004).
Student motivation is one factor that has and will
continue to plague faculty. Is the student interested in the
material and thus motivated to learn it? If the student is
interested and can see a purpose in learning the material
such that it is needed to be used on the job, the chances
are that he or she will learn it. The student will
subsequently demonstrate the command of that material
in a variety of ways including doing well on the job and
performing well in those subject areas on examinations
for licensure.

It is difficult to compare the performance of candidates in TRENDS IN ACADEMIA

one discipline with candidates in other disciplines on
licensure examinations. The examination statistics Generalized trends in the undergraduate academic
presented by each discipline are generally different for experience include: 1) general increase in the time spent
earning an undergraduate degree, 2) grade inflation, and
one or more of the following reasons:
3) decline in the number of hours that the typical student
spends in studying and preparing for classes (Engle, 2004
frequency of data compilation,
mixed data on the performance of first-time test a,b). The latter two items seem to present an interesting
takers compared to those who have tried multiple paradox. One might deduce either that students are
becoming more skilled or that less is being required of
times,
students during their academic experiences.
timing of examination in career development,
There is no known comprehensive review that
prerequisites required to take the examinations in
shows trends in degree requirements of undergraduate
terms of degree and/or experience,
Williams et al. - National Association of State Boards of Geology (ASBOG) Fundamentals Examination

375

Administration Date

Reliability Coefficients

Oct 92
Apr 93
Oct 93
Apr 94
Oct 94
Apr 95
Sep 95
Ar 96
Sep 96
Apr 97
Sep 97
Apr 98
Sep 98
Mar 99
Sep 99
Mar 00
Sep 00
Mar 01
Sep 01
Mar 02
Oct 02
Mar 03
Oct 03
Average

0.88
0.76
0.75
0.80
0.82
0.84
0.77
0.82
0.80
0.82
0.83
0.84
0.84
0.87
0.87
0.86
0.86
0.84
0.86
0.88
0.87
0.86
0.88
0.84

EXAMINATION DEVELOPMENT PROCESS

Figure 3. National Association of State Boards of

Geology FG Examination Estimated reliability
across administrations (October 1992 October
2003).

geological curriculum in United States colleges and

universities. Limited evidence, much of which is
uncataloged and anecdotal, suggests that many geology
(geoscience) departments are extremely concerned about
the decline in majors and are exploring ways of making
the major more attractive to students. In part, this
concern is motivated by a desire to insure departmental
survival. In 2003, the University of Connecticut moved to
eliminate the long-standing geology and geophysics
department for a number of reasons including the
decline in number of majors. Colleges and universities
are aware of the nation-wide decline in the numbers of
geology undergraduate majors that has occurred over
the past 25 years. In the early 1980s, there were more
than 35,000 undergraduate majors in geology. Currently
there are approximately 10,000 (AGI Web Site, 2004).
Much, but not this entire decline was precipitated by
changes in employment opportunities, particularly the
decline in hiring by the petroleum industry.
In an effort to make programs more attractive, some
college and university geology departments have
adjusted the degree requirements to require less rigorous
math and physics, have reduced the number of required
electives, and have eliminated the requirement for a
summer field camp. These changes seem to be
independent of the geographic region of the United
States. In addition to departmental actions taken to
address the decline in majors problems, there are
pressures, external to the department. For example, all
departments, including geology in the California State
University system are under increasing pressure to
reduce to 120 semester units the number of units
376

required for all undergraduate degrees. Most geology

undergraduate degrees in the California State University
system currently require more than 120 semester units,
some as many as 132 semester units. Given that the
majority of courses outside the discipline are mandated
by university requirements such as those for general
education, physical education, etc., it is likely that the
unit cuts must come from geology or supporting science
courses.

All
states
administer
licensure
examinations
simultaneously in the spring and fall of each year. The
Fundamentals of Geology (FG) Examination (110 items)
is administered during a 4-hour period. The FG Exam is
developed so that most candidates will have sufficient
time to complete all items in the examination. The exam
follows guidelines established in the Standards for
Educational and Psychological Testing (1999) published by
the American Educational Research Association, the
American Psychological Association, and the National
Council on Measurement in Education. The test
development process is designed to maximize the
fairness and quality of the examination as a measure of
minimum competency.
ASBOG conducts task analysis surveys (TAS) of the
profession every five years to maximize the relevance of
the examination for candidates seeking licensure as
professional geologists. The TAS is used to verify those
tasks performed by the profession related to public
protection. It is necessary that the task analysis be
repeated at 4-5 year intervals to ensure that the licensing
examinations developed from the analyses accurately
reflect the current practice of geology. A good example of
the rapid change in geologic practice has been the shift
from petroleum-dominated activity in the late 1970s and
early 1980s to the dominance of environmental activities
such as groundwater, groundwater contamination,
hazardous waste sites, etc. In less than ten years, the
practice of geology made a very dramatic shift. The
survey findings are used to develop test blueprints (test
specifications, content outlines) for constructing
examinations and writing questions. The test blueprints
list the geologic tasks and the number of questions for
each geologic task to be included in the FG Exam.
The subject matter tested on the FG Examination is
divided into nine subject areas or content domains: field
methods and remote sensing (28% of examination
questions), mineralogy, petrology, petrography and
geochemistry (14%), sedimentology, stratigraphy and
paleontology (10%), geomorphology (6%), structural
geology and tectonics (9%), geophysics and seismology
(4%), hydrogeology (25%), engineering geology (3%),
and mineral, petroleum and energy resources (1%)
(Figure 1).
ASBOG conducted the Task Analysis 2000 study
with the objective of identifying the relative importance
of the tasks performed by practicing geologists in the
USA and Canada. Consistent with the research
completed in 1995, the Task Analysis 2000 study
demonstrated that the tasks performed by geologists in
different states are very similar. These findings support
the feasibility of developing national examinations that
are fair to candidates from all regions of the country.
The results from this study were incorporated into
FG Exam beginning with the September 2000
administration (see Figure 1). The Task Analysis 2000
Questionnaire included 61 tasks distributed across nine
content domains, addressed 19 ethical issues that

Journal of Geoscience Education, v. 52, n. 4, September, 2004, p. 374-378

geologists encounter in the practice of the profession,

and incorporated questions about respondents
background characteristics and practice demographics.
ASBOG mailed 5,756 questionnaires to random samples
of licensed geologists in twenty-one states and nine
provinces (USA total = 4,200; Canada = 1,556). A total of
2,537 questionnaires were completed and returned (2,537
/ 5,756 = 44% return rate).
A committee of professional geologists from
throughout the United States serves as Subject Matter
Experts on the Council of Examiners (COE). Members of
the COE represent the profession in terms of geography,
ethnicity, gender, and area of practice. They supply the
expertise that is essential to develop fair and impartial
examinations for measuring competency within the
profession. By utilizing the expertise of individuals from
throughout the nation, ASBOG provides uniform exams
that are valid for a wide range of geographic regions and
professional practice settings. Members of the COE
attend two test development workshops annually. The
workshops are held shortly after the exam has been
administered so that COE Members can evaluate
candidates comments and statistical information on
each item.
The statistical analyses are useful for isolating items
that possess unusual statistical properties (e.g., very
difficult, negative correlations). Items that possess
negative correlations reveal that candidates with high
test scores did poorer on these items compared to
candidates with low test scores. The COE eliminates any
flawed items before scores are finalized and mailed to
candidates. Typically, two or three flawed questions
per examination administration are eliminated from the
scoring.
Reliability refers to the consistency or precision of
measurement that a test provides. This index ranges
from zero to one where .70 and above is generally
considered acceptable for licensing exams. The reliability
coefficient (sometimes referred to as coefficient alpha) is
based on the average correlation of items within a test
(i.e., internal consistency of test items). It is reasonable to
assume that items within a test will positively correlate
with each other because each of the test items is
developed with the intent of assessing, to a certain
extent, the same construct (i.e., competency as a
professional geologist).
All exams contain some measurement error.
However, the amount of error can be reduced by
improving the quality and accuracy of items in the
exams. As measurement error decreases, the reliability
coefficient increases. Typically, the reliability coefficient
will be higher if the test is composed of quality items that
discriminate well. Figure 3 displays the reliability
coefficients for the FG Exam from October 1992 through
October 2003. The estimated reliability for the FG Exam
has repeatedly exceeded .70 (average reliability
coefficient = .84). The reliability coefficients have
remained remarkably high, with a slight increase over
the last five years (average reliability coefficient = .87).
The grading approach used by ASBOG and many
state boards is the criterion-referenced method. This
approach makes use of licensed, subject-matter experts
who collectively evaluate each question as to its
difficulty and estimate the percentage of those minimally
qualified candidates who will get the question correct on
an examination (Livingston and Zieky, 1982). A subject
matter expert is an individual who by education and
work experience has extensive knowledge about a
particular segment of the discipline. Generally, these

individuals are licensed in the particular profession. For

example, in the case of geology, this might be an
individual particularly knowledgeable in hydrogeology.
These individuals contribute to the preparation,
evaluation, and review of questions used on the licensing
examinations. Making use of this input for each question,
an overall pass point is established to reflect minimum
competency in the profession.
The grading of the examinations to determine
whether a candidate has earned a satisfactory score is a
complex issue. Those faced with using performance on
the examinations as a measure of determining if a
candidate has been successful have a number of
challenges with which to deal. These include establishing
the appropriate pass point and determining that
uniformity of examination difficulty is maintained
between exam administrations. Although efforts are
made to keep examinations of equal difficulty, there is
some variation in examination difficulty from
administration to administration. Thus unless care is
exercised, some candidates could have examinations
more difficult than their colleagues examinations taken
at a different time. This would not be an equitable or fair
situation.
Candidates raw scores are calculated by summing
the number of correct responses for all scored items in
the exam. Credit is given for correct responses; no points
are received for incorrect responses, and points are not
subtracted for incorrect answers. A scaled score of 70 has
been established as a standard of minimum competency,
and 100 is the highest score possible.
Different forms of the examination are equated,
using statistical adjustments, so that passing scores on all
forms of the examination reflect the same standard of
minimum competency. This approach minimizes the
impact of variations in examination difficulty from
administration to administration. Thus, for example, on
an examination found more difficult, a score of 65 %
correct might be passing while on an easier
examination, one might need a 73 % correct. Generally,
the scores are reported to candidates as scaled scores
with a scaled score of 70 reflecting the minimum passing
score on all forms of the exam. This practice avoids
confusing candidates if (when) they compare their
results to those of their colleagues obtained at different
times (Williams, 2002, Warner, 2002).
Candidates who fail the exam receive feedback on
their performance levels for the content domains listed in
the FG Test Blueprint. A + indicates acceptable
performance in a particular content domain whereas a
- shows substandard performance. The feedback is
designed to assist candidates in evaluating their
proficiency levels in preparation for subsequent exams.

SAMPLE QUESTIONS FROM

FUNDAMENTALS EXAMINATION
1.

According to the Unified Soil Classification, a soil

described as a GW is a (an):
a) Well-graded gravel or gravel-sand mixture, with no or
little fines
b) Poorly- graded gravel or gravel-sand mixture, with no
or little fines
c) coarse clayey gravel
d) organic silt of low plasticity
2.

Which one of the following minerals dissolves into

soluble ions without residue?
a) kaolinite

Williams et al. - National Association of State Boards of Geology (ASBOG) Fundamentals Examination

377

b) pyrite
c) selenite
d) orthoclase
3.

A phaneritic igneous rock composed of orthoclase,

oligoclase, biotite, hornblende, and quartz is:
a) monzonite
b) syenite
c) latite
d) granodiorite
4.

What group of commonly occurring sedimentary

deposits forms by precipitation of salts from
land-locked bodies of concentrated solutions or
brines?
a) sulfuric sedimentary rocks
b) organic sedimentary rocks
c) evaporitic sedimentary rocks
d) phosphatic sedimentary rocks
5.

four-year curriculum. To be successful, the candidate

generally needs to answer correctly about 60% of the
questions. Members of the profession, based upon what
practicing professionals see as the skills and knowledge
needed by an individual to be successful in the
profession, design the examination. There seems to be a
disconnect between what the students know upon
graduation and what the profession believes they should
know. Many reasons may be contributing to this
including: the subject matter being covered in the
academic courses is not aligned as it should be toward
the practice of geology; faculty are not demonstrating
how the material can be applied in the profession;
students do not see the value of retaining the information
for application in their careers, etc. Students may be
becoming disheartened to find that while they believe
they have been educated appropriately to take their
positions in the profession only to find that almost half of
them have problems in being able to compete
successfully for professional positions.

A map at the scale of 1:24,000 compared to a map at

the scale of 1:62,500 is:
REFERENCES
a) a smaller scale map
b) a larger scale map
American Geological Institute (AGI), 2004, http://www.
c) larger scale or smaller scale dependent upon the units
agiweb.org/.
of measurement
American Educational Research Association, 1999,
d) larger scale or smaller scale dependent upon the
Standards for Educational and Psychological
ground area shown
Testing, American Psychological Association, and
National Council on Measurement in Education,
6. An aerial photograph taken with a camera having a
Washington, DC.
focal length of 6 inches flying 10,000 feet above the ASBOG, 1998, Annual Meeting Book (1998) Charleston,
datum has a scale of:
South Carolina, National Association of State Boards
a) 1:10,000
of Geology, Columbia, South Carolina.
b) 1:20,000
ASBOG Website, 2004, www.asbog.org
c) 1 inch = 10,000 feet
Ebel, Robert L. and Frisbie, David A., 1991, Essentials of
d) Scale cannot be determined from the data given
Educational Measurement (5th Ed.) Englewood
Cliffs, New Jersey: Prentice-Hall, Inc.
SUMMARY
Engle, Shaena, 2004a, Degree Attainment Rates at
Colleges and Universities: UCLA Higher Education
The purpose of this paper is to document a situation, not
Research Institute, www.gseis.ucla.edu/heri/darcu
to provide solutions. Developing solutions is beyond the
_pr.html.
scope of this work for it is the authors belief that before Engle, Shaena, 2004b, Political Interest on the Rebound
solutions can be developed, an identification of the
Among the Nations Freshmen: UCLA Higher
problem, if there is one, must occur. In the past two
Education Research Institute, www.gseis.ucla.edu/
decades, fewer and fewer students are enrolling and
heri/darcu_pr.html.
graduating with majors in the geosciences. The reasons Livingston, Samuel A. and Zieky, Michael J., 1982,
for this trend are not clear, but changes in the
Passing Scores: A Manual for Setting Standards of
employment sectors from petroleum with its large
Performance on Educational and Occupational
demands for personnel to the reduced number of
Tests. Princeton, New Jersey: Educational Testing
personnel required by environmental geology are
Service.
certainly contributing factors. Competition from National Association of State Boards of Geology, 2002,
employment arenas considered more lucrative such as
http://www.asbog.org/
computer science, engineering, and information systems TEST Inc., 2001, National Association of State Boards of
is also a factor. Geoscience departments have made shifts
Geology Fundamentals of Geology and The Practice
in curriculum to accomplish two things; 1) update the
of Geology Forms 0103-March 2001 Summary
curriculum to reflect the needs of the discipline and the
Report, National Association of State Boards of
technological advances and 2) attempt to attract and
Geology, Columbia, South Carolina.
retain the scarce geoscience major. Graduating students National Council of Examiners for Engineering and
are finding the majority of employment opportunities
Surveying, 2004, http://www.ncees.org/
are in the applied areas of geology particularly Warner, Jack, 2002, personal communication, TEST Inc.,
hydrogeology, engineering geology, and environmental
Aurora, Colorado.
geology. Many positions in these areas require state Williams, John W., 2002, Examinations in the Licensure
licensing in order to advance and to assume more
Process for Geologists: Association of Engineering
responsible job assignments. To earn state licensing as
Geologists Annual Meeting, Reno, NV.
geologists, almost all are required to take a written Williams, John W., Warner, Jack L., and Warner, Steven
examination.
P., 2003, Subject Area Knowledge Measured by
Performance on these licensing examinations has
Scores on the National Association of State Boards of
been somewhat discouraging. Only about 60 percent of
Geology (ASBOG) Fundamentals Examination and
candidates are successful on the fundamentals
the Implications for Academic Preparation (abs.):
examination that is designed to test the knowledge and
Association of Engineering Geologists Annual
skills that the candidate has upon graduation from a
Meeting, Vail, CO.
378

Journal of Geoscience Education, v. 52, n. 4, September, 2004, p. 374-378