What is validity?

Without a doubt, one of the most important criteria that is used to evaluate the quality of a test has
got to be the validity of that test. In definition, validity has been defined as the measure of whether
or not the test being presented lives up to the expectations and requirements of what It does claim to
measure. This measure is the intended cause for a test. A test will be considered to be highly valid if
the items present in the test are closely linked to the initial intended focus of the test (Western,
2008).
Construct validityThis is whether the measurements of a variable in a study behave in exactly the same way as the
variable itself. This involves examining past research regarding different aspects of the same
variable. The use of construct validity in psychology is examined by Cronbach and Meehl (1955)

Content Validity
In as much as there are quite a number of types of validity, content validity has got to be the most
important type of them all (PTI, 2006). For the certification and licenser programs, this is most
probably the most important type of validity that they take into consideration first. Content validity
is considered most important because of the emphasis that it dies place on the test items and the job
related tasks (Brown, 2011). It is a logical process whereby the connections that do exist between
these two points of emphasis are established and examined carefully for their relevancy. There are a
number of factors that are looked at when establishing the content validity.
What is reliability?
Reliability refers to the consistency of a measure. A test is considered reliable if we get the same
result repeatedly. For example, if a test is designed to measure a trait (such asintroversion), then
each time the test is administered to a subject, the results should be approximately the same.
Unfortunately, it is impossible to calculate reliability exactly, but it can be estimated in a number of
different ways.
Importance of validity?
Validity is important because it measures the accuracy of results within a study and therefore the
degree to which you can make assumptions, correlations and relationships from data. Evidence
cannot be reliable if it has poor validity, however if it has poor reliability it cannot be valid.
Validity, reliability, resources and manageability. To decide the best way of conducting assessment

in a particular case it is necessary to consider the properties of possible tools in relation to the
purposes and uses intended for the assessment results. One obvious desirable property is that any
assessment should be valid for its purpose; that it assesses what it is intended to assess. Another is
that it should provide reliable, or trustworthy, data. But there are also other matters to be taken into
account; in particular, in view of the interdependence of the various system components, the impact
on other assessment practices, on the curriculum and on pedagogy. Further, the use of resources
assessment can be costly, both in terms of monetary.
Case study
Introduction
As Ford Motor Company continues to seek ways to improve its current hybrid vehicles, the
company will conduct a statistical analysis of the results of the questionnaires returned by its
customers. Through this analysis, Ford Motor Company will be able to identify challenges that the
company faces as well as the steps needed to reduce these challenges and improving both the
companys product and image.
Potential Challenges
Potential challenges to the validity and reliability of the research include variations in gas prices,
the varying quality of inspections, and highway verses country mileage. Gas prices vary depending
on the state of the economy and location. Prices of gasoline per gallon change consistently by as
much as two dollars within the year. This could sway slightly the end result of miles per dollar each
subject receives. The number of inspections that a customer receives within the year could be due to
the quality of scheduled maintenance they have received. If potential problems were not noticed at
first inclination, then they could develop into more serious complications- in this case analyzed as
manufacturing problems. Most cars receive more mileage during stents of open road. It is in the city
with constant starting and stopping along with slower mph that lessen miles attained by the car. For
the assumptions of statistics to be true, all of these would need to be constant and analyzed under
the same conditions.
Steps to Minimize these Challenges
The potential challenges posed by our research question, data, and analysis are unfortunately
beyond our control. We have no control over the state of the economy and location in which our
research is conducted. Furthermore, we have no control over gas price variations, quality of
inspections, and the consumers country mileage versus their highway mileage. So our best mode of
defense to minimize these challenges is to broaden the research area and conduct multiple location
research. This will allow us to have a more accurate accounting of various locations and be able to

apply the results to similar locations. No vehicle can be 100% perfect for the car owner, and the car
owners usage of the vehicle plays a major role in many aspects of vehicle efficiency and
economics. Therefore, no changes need to be made to the research question or questionnaire.
Instead, the research just needs to be expanded for accuracy and to increase reliability and validity
of the data.
General Response Types
A lot of thought goes into determining the most appropriate measurement scale to be used in
research. The scales are placed into one of the four general response types: categorization, ranking,
sorting, or rating (Cooper & Schindler, 2011). In our research and measurement scale, we went with
the questionnaire method. This method allows us to gather as much information as possible from
many consumers. Furthermore, it provides the information to us in an organized fashion to make
placing the response to the questionnaire underneath the appropriate general response types. The car
year, make, and model falls underneath the sorting response type. The most valued feature of the
owners vehicles falls underneath the ranking general response type. The scheduled maintenance
performed, car trouble, miles driven per week, times stopped for gas per week, and price to fill up
the vehicle falls underneath the categorization general response type. And lastly, the owners
satisfaction with their current hybrid vehicle falls underneath the rating general response type.
Classify your findings in order of power
As Ford Motor Corporation analyzes the results of its questionnaire, it will be able to identify which
variables have the most impact on current hybrid vehicle owners. The current questionnaire
identifies eight different variables for customers to rate. Of the eight questions asked, the questions
concerning mileage and maintenance are weighted higher as they are more important to current and
potential customers.
Each questions is weighted based on the responses received and ranked. By doing this, Ford is able
to apply additional resources or efforts to questions that the company feels it should focus on to
increase the marketability of their vehicles. To further aid in increasing their marketability, Ford
will utilize a quadratic model to verify the results of the questionnaires returned by its customer. Of
the measurement scales utilized, nominal, ordinal, interval and ratio scales, the ratio scales which
incorporate all of the powers of the previous scales plus the provision for absolute zero or origin
(Cooper & Schindler, 2011, p. 277) will provide the variables utilized in the quadratic model.
Specifically, a complete second-order model will allow Ford to identify mileage and maintenance as
independent variables as the company works towards improving their current vehicles.

Evaluate processes to ensure validity and reliability

Within the conduct of inquiry itself, verification strategies that ensure both reliability and validity of
data are activities such as ensuring methodological coherence, sampling sufficiency, developing a
dynamic relationship between sampling, data collection and analysis, thinking theoretically, and
theory development.
1. First, the aim of methodological coherence is to ensure congruence between the research question
and the components of the method. The interdependence of qualitative research demands that the
question match the method, which matches the data and the analytic procedures. As the research
unfolds, the process may not be linear. Data may demand to be treated differently so that the
question may have to be changed or methods modified. Sampling plans may be expanded or change
course altogether. The fit of these components with data to meet the analytic goals must be
coherent, with each verifying the previous component and the methodological assumptions as a
whole.
2. Second, the sample must be appropriate, consisting of participants who best represent or have
knowledge of the research topic. This ensures efficient and effective saturation of categories, with
optimal quality data and minimum dross. Sampling adequacy, evidenced by saturation and
replication (Morse, 1991), means that sufficient data to account for all aspects of the phenomenon
have been obtained. Seeking negative cases is essential, ensuring validity by indicating aspects of
the developing analysis that are initially less than obvious. By definition, saturating data ensures
replication in categories; replication verifies, and ensures comprehension and completeness.
3. Third, collecting and analysing data concurrently forms a mutual interaction between what is
known and what one needs to know. This pacing and the iterative interaction between data and
analysis (as discussed earlier) is the essence of attaining reliability and Morse, Barret, Mayan,
Olson, & Spiers,
4. The fourth aspect is thinking theoretically. Ideas emerging from data are reconfirmed in new data;
this gives rise to new ideas that, in turn, must be verified in data already collected. Thinking
theoretically requires macro-micro perspectives, inching forward without making cognitive leaps,
constantly checking and rechecking, and building a solid foundation.
5. Lastly, the aspect of theory development is to move with deliberation between a micro

perspective of the data and a macro conceptual/theoretical understanding. In this way, theory is
developed through two mechanisms:
As an outcome of the research process, rather than being adopted as a framework to move the
analysis along; and As a template for comparison and further development of the theory. Valid
theories are well developed and informed, they are comprehensive, logical, parsimonious, and
consistent 8; (see Glaser 1978; Morse 1997).
References
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey, CA:
Brooks/Cole.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16,
297-334.
Gulliksen, H. (1950). Theory of mental tests. New York:Wiley.
Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability.
Psychometrika, 2, 151-160.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill.

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (1985). Standards for educational
and psychological testing. Washington, DC: Authors.
Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer & H. I. Braun
(Eds.), Test validity.Hillsdale, NJ: Lawrence Erlbaum.
Brennan, R. (2001). An essay on the history and future of reliability from the
perspective of replications.Journal of Educational Measurement, 38, 295-317.
Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.). Educational
Measurement (2nd Ed.). Washington, D. C.: American Council on Education.
Cronbach, L. J. (2004). My current thoughts on Coefficient Alpha and successor
procedures. Educational and Psychological Measurement, 64, 391-418.
Cronbach, L. J. & Quirk, T. J. (1976). Test validity. In International Encyclopedia of
Education. New York: McGraw-Hill.
Goodenough, F. L. (1949). Mental
applications. New York: Rinehart.

testing:

Its

history,

principles,

and

Hunter, J. E.; & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error

and bias in research findings. Newsbury Park: Sage Publications.
Kane, M. (2001). Current concerns in validity theory. Journal of educational
Measurement, 38, 319-342.

Lacity, M.; & Jansen, M. A. (1994). Understanding qualitative data: A framework of

text analysis methods. Journal of Management Information System, 11, 137-160.
Li, H. (2003). The resolution of some paradoxes related to reliability and
validity. Journal of Educational and Behavioral Statistics, 28, 89-95.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences
from persons' responses and performance as scientific inquiry into scoring
meaning. American Psychologist, 9, 741-749.
Messick, S. (1998). Test validity: A matter of consequence. Social Indicators
Research, 45, 35-44.
Mislevy, R. (2004). Can there be reliability without reliability? Journal of
Educational and Behavioral Statistics, 29, 241-244.
Moss, P. A. (1994). Can there be validity without reliability? Educational Researcher,
23, 5-12.
Osterlind, S. J. (1983). Test item bias. Newbury Park: Sage Publications.
Parkes, J. (2000). The relationship between the reliability and cost of performance
assessments.Education
Policy
Analysis
Archives,
8. Retrieved
from http://epaa.asu.edu/epaa/v8n16/
Pedhazur, E. J.; & Schmelkin, L. P. (1991). Measurement, design, and analysis: An
integrated approach.Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
Polkinghorne, D. E. (1988). Narrative knowing and the human sciences. Albany:
State University of New York Press.
Salvucci, S.; Walter, E., Conley, V; Fink, S; & Saba, M. (1997). Measurement error
studies at the National Center for Education Statistics. Washington D. C.: U. S.
Department of Education
Thompson, B. (Ed.) (2003). Score reliability: Contemporary thinking on reliability
issues. Thousand Oaks: Sage.
Yu, C. H. (2005). Test-retest reliability. In K. Kempf-Leonard (Ed.). Encyclopedia of
Social Measurement, Vol. 3 (pp. 777-784). San Diego, CA: Academic Press.