Master of System Engineering

Faculty of Engineering
Gadjah Mada University
Research Methodology
and
Project Proposal Preparation
Adhy Kurniawan
Faculty of Engineering
Gadjah Mada University
Adhy Kurniawan
1987-1990 SMA 3 Semarang
1990-1991 Fac. Of Economy, Diponegoro
University, Semarang
S1(1991-1996) Civil Engineering Dept.
Gadjah Mada Univ.
S3(1998-2003) Swiss Federal Institute of
Technology Lausanne (EPFL), Swiss
Post Doct (nov.2005-sept.2006) Kyoto
University, Japan
My Goals for Course

That each of you develop an

intuition for the fundamental
principles of research methodology
That we have an enjoyable
semester learning together
Lecture and Homework

Lecture
Presentation and discussion

Homework
Your chance to practice using the
concepts presented in class
Teamwork vs. Individual work?
References
All of literature concerning: Res Met
Marczyk, DeMatteo, Festinger. 2005,
Essentials of Research Design and
Methodology, John Wiley and Sons.
Day and Gastel, 2006, How to write and
Publish a Scientific Report, Greenwood
Press
Metodologi Riset, Etc.
List of students
Alif Ardy Saputra, Geodesi UGM
Anik FR, TL, ITB
Ashri Uswatun, TFisika,UGM
Ayi Fajarwati, TL, ITB
Corry Agustina, Perenc Wil, TA, UGM
Dwi Astuti, TKimia, UGM
Elva Nur , TF, UGM
Erika Kezia, TL, ITB
Fitri Wijayanti, Fisika, UNS
I Nyoman Kusuma, TF, UGM
Ihsan Hasan, T Industri, UII,
Ihwan Ghazali, T Industri, UAD
Iin Lestari, TL, ITB
M Sony Abertiawan, TL, ITB
Maria Auliana, T Sipil, UGM
Norma Pradipta, TArsitektur, UGM
Satrya Alrizki, TGeofisik, ITB
Tatag Lindu Bhakti, TFisika, UGM
Contents
The aims of research,
the research topic,
title and research problem,
literature review,
research design: population and sampling types, types
of quantitative research designs, validity of
conclusions, data-collecting methods and
measuring instruments in quantitative research,
qualitative research designs,
data analysis and interpretation of results,
report writing and the research proposal,
ethical consideration on research.
 OVERVIEW OF SCIENCE AND THE
SCIENTIFIC METHOD
science can be defined as a methodological and
systematic approach to the acquisition of new
knowledge.
This definition of science highlights some of the
key differences between how scientists and
nonscientists go about acquiring new
knowledge.
Specifically, rather than relying on mere casual
observations and an informal approach to learn
about the world, scientists attempt to gain new
knowledge by making careful observations and
using systematic, controlled, and
methodical approaches (Shaughnessy &
Zechmeister, 1997).
Shaughnessy, J. J., & Zechmeister, E. B. (1997). Research methods in
psychology (4th ed.). Boston: McGraw Hill.
 In addition, scientific knowledge is not
based on the opinions, feelings, or
intuition of the scientist.
Instead, scientific knowledge is based on
objective data that were reliably obtained
in the context of a carefully designed
research study.
In short, scientific knowledge is based
on the accumulation of empirical
evidence (Kazdin, 2003a)
Kazdin, A. E. (2003a). Methodology: What it is and why it is so important. In A.
E. Kazdin ( Ed.), Methodological issues and strategies in clinical research (3rd
ed., pp. 522). Washington, DC: American Psychological Association.
The defining characteristic of scientific
research is the scientific method .
First described by the English
philosopher and scientist Roger Bacon in
the 13th century, it is still generally
agreed that the scientific method is
the basis for all scientific
investigation.
The scientific method is best thought of
as an approach to the acquisition of new
knowledge, and this approach effectively
distinguishes science from nonscience.
The Scientific Method
The development of the scientific method is
usually credited to Roger Bacon, a philosopher
and scientist from 13th-century England,
although some argue that the Italian scientist
Galileo Galilei played an important role in
formulating the scientific method.
Later contributions to the scientific method were
made by the philosophers Francis Bacon and
Ren Descartes.
Although some disagreement exists
regarding the exact characteristics of the
scientific method, most agree that it is
characterized by the following elements:
Empirical approach
Observations
Questions
Hypotheses
Experiments
Analyses
Conclusions
Replication
 Empirical Approach
The scientific method is firmly based on the empirical
approach. The empirical approach is an evidence-based
approach that relies on direct observation and
experimentation in the acquisition of new knowledge (see
Kazdin, 2003a).

In the empirical approach, scientific decisions are made based

on the data derived from direct observation and
experimentation.
Contrast this approach to decision making with the way that
most nonscientific decisions are made in our daily lives.
For example, we have all made decisions based on feelings,
hunches, or gut instinct. Additionally, we may often reach
conclusions or make decisions that are not necessarily based
on data, but rather on opinions, speculation, and a hope for
the best.
The empirical approach, with its emphasis on direct,
systematic, and careful observation, is best thought of as the
guiding principle behind all research conducted in accordance
with the scientific method.
Observations
An important component in any scientific investigation is
observation. In this sense, observation refers to two
distinct conceptsbeing aware of the world
around us and making careful measurements.
Observations of the world around us often give rise to the
questions that are addressed through scientific research.
For example, the Newtonian observation that apples fall
from trees stimulated much research into the effects of
gravity. Therefore, a keen eye to your
surroundings can often provide you with many
ideas for research studies.
Questions
After getting a research idea, perhaps from making
observations of the world around us, the next step in the
research process involves translating that research idea
into an answerable question.
The term answerable is particularly important in this
respect, and it should not be overlooked.
It would obviously be a frustrating and ultimately
unrewarding endeavor to attempt to answer an
unanswerable research question through scientific
investigation.
It is therefore important to formulate a research
question that can be answered through available
scientific methods and procedures.
Hypotheses
The next step in the scientific method is coming up with a
hypothesis, which is simply an educatedand
testableguess about the answer to your research
question.
A hypothesis is often described as an attempt by the
researcher to explain the phenomenon of interest.
Hypotheses can take various forms, depending on the
question being asked and the type of study being
conducted.
A key feature of all hypotheses is that each must make a
prediction.
Remember that hypotheses are the researchers attempt
to explain the phenomenon being studied, and that
explanation should involve a prediction about the
variables being studied.
These predictions are then tested by gathering and
analyzing data, and the hypotheses can either be
supported or refuted on the basis of the data.
 Two types of hypotheses with which you should be
familiar are
the null hypothesis
and the alternate (or experimental) hypothesis.
The null hypothesis always predicts that there will be no
differences between the groups being studied.
By contrast, the alternate hypothesis predicts that there
will be a difference between the groups.
For example,
the null hypothesis would predict that the
exercise group and the no-exercise group will not
differ significantly on levels of cholesterol.
The alternate hypothesis would predict that
the two groups will differ significantly on
cholesterol levels.

Homework: Individual
Please try to find one example. About the null
Experiments
After articulating the hypothesis, the next step involves
actually conducting the experiment (or research
study).
For example, if the study involves investigating the effects
of exercise on levels of cholesterol, the researcher would
design and conduct a study that would attempt to
address that question.
As previously mentioned, a key aspect of conducting a
research study is measuring the phenomenon of interest
in an accurate and reliable manner.
In this example, the researcher would collect data on the
cholesterol levels of the study participants by using an
accurate and reliable measurement device.
Then, the researcher would compare the cholesterol
levels of the two groups to see if exercise had any
effects.
 Accuracy vs. Reliability
When talking about measurement in the context of research,
there is an important distinction between being accurate and being
reliable.
Accuracy refers to whether the measurement is correct, whereas
reliability refers to whether the measurement is consistent.
An example may help to clarify the distinction.
When throwing darts at a dart board, accuracy refers to
whether the darts are hitting the bulls eye (an accurate dart
thrower will throw darts that hit the bulls eye).
Reliability, on the other hand, refers to whether the darts
are hitting the same spot (a reliable dart thrower will throw
darts that hit the same spot).
Therefore, an accurate and reliable dart thrower will
consistently throw the darts in the bulls eye. As may be
evident, however, it is possible for the dart thrower to be
reliable, but not accurate.
For example, the dart thrower may throw all of the darts in
the same spot (which demonstrates high reliability), but that
spot may not be the bulls eye (which demonstrates low
accuracy).
Analyses
After conducting the study and gathering the
data, the next step involves analyzing the data,
which generally calls for the use of statistical
techniques.
The type of statistical techniques used by a
researcher depends on the design of the study,
the type of data being gathered, and the
questions being asked.
It is important to be aware of the role of
statistics in conducting a research study.
In short, statistics help researchers minimize
the likelihood of reaching an erroneous
conclusion about the relationship between the
variables being studied.
Conclusions
After analyzing the data and determining
whether to reject the null hypothesis, the
researcher is now in a position to draw some
conclusions about the results of the study.
For example, if the researcher rejected the null
hypothesis, the researcher can conclude that
the phenomenon being studied had an effect
a statistically significant effect, to be more
precise.
If the researcher rejects the null hypothesis in
our exercise-cholesterol example, the researcher
is concluding that exercise had an effect on levels
of cholesterol.
It is important that researchers
make only those conclusions that
can be supported by the data
analyses.
Going beyond the data is a cardinal
sin that researchers must be careful
to avoid.
Replication
One of the most important elements of
the scientific method is replication.
Replication essentially means
conducting the same research study a
second time with another group of
participants to see whether the same
results are obtained.
The same researcher may attempt to
replicate previously obtained results, or
perhaps other researchers may
undertake that task.
 Replication illustrates an important point about
scientific researchnamely, that researchers
should avoid drawing broad conclusions based
on the results of a single research study
because it is always possible that the results of
that particular study were an aberration.
In other words, it is possible that the results of
the research study were obtained by chance or
error and, therefore, that the results may not
accurately represent the actual state of things.
However, if the results of a research study are
obtained a second time (i.e., replicated), the
likelihood that the original studys findings were
obtained by chance or error is greatly reduced.
What are the three general
goals of scientific research?
Answer:

description,
prediction,
and understanding/explaining
What Exactly is Research?

we will focus on two of the most

common types of research
correlational research
and experimental research
Correlational research:
In correlational research, the goal is to determine whether
two or more variables are related. (By the way, variables
is a term with which you should be familiar.
A variable is anything that can take on different values,
such as weight, time, and height.)
For example, a researcher may be interested in
determining whether age is related to weight. In this
example, a researcher may discover that age is indeed
related to weight because as age increases, weight also
increases. If a correlation between two variables is strong
enough, knowing about one variable allows a researcher
to make a prediction about the other variable.

It is important to point out, however, that a correlation

or relationshipbetween two things does not necessarily
mean that one thing caused the other.To draw a cause-
and-effect conclusion,
researchers must use experimental research.
.
Experimental research:
In its simplest form, experimental research involves
comparing two groups on one outcome measure to test
some hypothesis regarding causation.
For example, if a researcher is interested in the effects of
a new medication on headaches, the researcher would
randomly divide a group of people with headaches into
two groups.
One of the groups, the experimental group, would receive
the new medication being tested.
The other group, the control group, would receive a
placebo medication (i.e., a medication containing a
harmless substance, such as sugar, that has no
physiological effects).
Experimental research:

Besides receiving the different medications, the groups

would be treated exactly the same so that the research
could isolate the effects of the medications. After
receiving the medications, both groups would be
compared to see whether people in the experimental
group had fewer headaches than people in the control
group.
Assuming this study was properly designed (and properly
designed studies will be discussed in detail in later
chapters), if people in the experimental group had fewer
headaches than people in the control group, the
researcher could conclude that the new medication
reduces headaches.
Task

Compose your own brief research

proposal.
Try to determine your research
topic for MST final project
Format:
1. In MS Word
2. In Power point
Task/assignment next week

Review 1 International Publication

(Journal, Conference paper,etc)
related to Renewable energy
Compose the summarize of your
review
Format:
1. In MS Word
2. In Power point
Purpose of the research proposal

1. To inform the reader of nature of your

proposed research.
What is the problem?
What is its extent?
2. To convince the reader, especially
supervisors and reviewers, of the value
of your proposed research.
Is this project worth the time
and money?
Will it make a difference to the
world?
Purpose of the research proposal

3. To demonstrate your expertise and

competency in a particular area of study.
Do you have the qualifications to conduct
this research?
Have you informed yourself of the
existing theory and data relevant to your
topic?
Do you have the
necessary skills to
conduct the research?
Purpose of the research proposal
4. To plan the research project and provide a
step-by-step guide to the tasks necessary
for its completion.

What are the key stages of the work?

What are the priorities?
How do the various components fit together?

5. To request support from individuals and

agencies who provide supervision,
oversight or funding for the research project.

What kinds of support does the project need?

Are all participants properly protected?
Purpose of the research proposal

6. To contract with the agencies

and individuals involved, including
supervisors, foundations and
participants in the research team.
How will tasks be assigned and
resources expended?
What does each contribute
to the collective endeavor?
First things first

1. Basics
2. Topic ideas
3. Typical methodologies
4. Common pitfalls
5. Getting started and putting it all
together
6. Questions/discussion
 Basic steps of a research
project
Find a topicWhat, When
Formulate questionsWhat, Why
Define populationWho, When
Select design & measurementHow
Gather evidenceHow
Interpret evidenceWhy
Tell about what you did and found out
Selecting a Research Topic

What are some considerations

when selecting a research topic?
 Considerations in Selecting a
Topic

Personal interest / Passion

Importance / Contribution to the field
Newness / Relevance
Feasibility
Tradeoff between rigor and practicality
Time constraints
Ethical constraints
Organizational support
Economic factors
Availability of Subjects
 Sources of Research Topics

Peer-reviewed journals in your field

Personal experiences
Work setting experiences
Existing literature
Recommendations for future research
 Refining Your Topic

Refinement needed for effective and

efficient research
Narrow your topic
Identify a theoretical framework
Specifically and unambiguously define
terms
State research questions and
hypotheses
 Refining Your Topic (contd)

A literature review will help you

See if your idea has been tried
Include all relevant constructs
Select instruments
Anticipate common problems
Components of a Concept
Paper
Title page
Introduction
Nature of the Problem
Background and Significance of the
Problem
Preliminary Literature Review
Initial Research Question or
Questions
Components of a Concept
Paper (contd)
Brief Description of Methodology
and Research Design
Anticipated Outcomes
Timeline
References
The Literature Review
 What is a Literature
Review?
According to Creswell (2005), a
review of the literature is a written
summary of journal articles, books
and other documents that describes
the past and current state of
information, organizes the literature
into topics and documents a need
for a proposed study. (pp. 79)
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating
Quantitative and Qualitative Research
Focusing on Empirical Research

What does Empirical Mean?

Primary Sources
Original Research Article

Secondary Sources
Newspapers
Book chapters
Television/Radio
Magazines
Wikepedia
Empirical Research
All empirical research is inherently
flawed
Limitations
1. Sampling
Generalizability
Representative
2. Measurement
Measurement Error
Social Desirability
3. Problem Identification
Grasping the Whole Problem
Literature Reviews

Well-written analytical narrative that

brings a reader up-to-date on what
is known on a given topic, but also
provide fresh insights that advance
knowledge
Resolve conflicts between studies
Identify new ways to interpret
research results
Creating a path for future research
Anecdotal Reports

A description of an event or
experience that happened to be
noticed
No control
No comparison
Review of Key Elements of
Previous Definition

The LR is a summary of research:

It is not a list of found research
but a coherent and articulate
account of past and current
research findings
Suggestion: read 2 or 3 LRs in order
to become familiar with summary
styles
 Review of Key Elements of
Previous Definition (contd)

The sources typically are journal articles, books

and other documents that describe past and
present status of research in a given field:
The LR should be exhaustive and as
current as possible.
How many articles?
There is no set number. As long as the search
is exhaustive and focused on the research
topic, the review will be acceptable.
 Review of Key Elements of
Previous Definition (contd)

How far back should one search?

A reasonable and widely accepted
timeframe includes research conducted
during the past 10 years. Important
studies (i.e., studies that had a
significant impact on the field of study)
should also be mentioned even if these
go beyond the mentioned timeframe.
 Review of Key Elements of
Previous Definition (contd)

The LR should be organized:

The review should not only be coherent, but
should organize the studies reviewed under
themes or topics.
The reviewer is a guide and should be able
to provide readers with an in-depth and
current status of research in a given area.
This aspect is essential for readers to
understand what the reviewer found during
the search.
 Review of Key Elements of
Previous Definition (contd)

The LR should document the need for a

proposed study:
Studies should not duplicate research that
has been already done.
Even in cases when research is duplicated
(replicated is the appropriate term), one is
responsible for documenting the need for
replication, e.g., need to explore the same
methodology with a different group or
population, or need to change methodology
with the same group.
 Creswells 5 steps to Conduct a
Literature Review

Step 1: Identify Key Terms or

Descriptors
Extract key words from your
title (remember, you may decide
to change the title later)
Use some of the words other
authors reported in the
literature

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Step 1: Identify Key Terms or
Descriptors (contd)
Use the Thesaurus of ERIC
Descriptors to look for terms
that match your topic: go to
www.eric.ed.gov and in Search
select Descriptors (from
Thesaurus)
Scan both electronic and
library journals from the past
10 years and look for key
terms in the articles
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Creswells 5 steps to Conduct a
Literature Review (contd)

Step 2: Locate Literature

Use academic libraries, do not limit
your search to an electronic search
of articles
Use primary and secondary sources.
A primary source is research
reported by the researcher that
conducted the study. A secondary
source is research that summarizes
or reports findings that come from
primary sources
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Step 2: Locate Literature
(contd)
It is best to report mostly
primary sources (p. 82)
Search different types of
literature: summaries,
encyclopedias, dictionaries and
glossaries of terms, handbooks,
statistical indexes, reviews
and syntheses, books, journals,
indexed publications,
electronic sources, abstract
series, and databases
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Creswells 5 steps to Conduct a
Literature Review (contd)
Step 3: Critically Evaluate and
Select Literature
Rely on journal articles
published in national journals
Prioritize your search: first
look for refereed journal
articles, then, non-refereed
articles, then books, then
conference papers,
dissertations and theses and
then papers posted to websites
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Step 3: Critically Evaluate and
Select Literature (contd)
Look for research articles and
avoid as much as possible
opinion pieces
Blend qualitative and
quantitative research in your
review

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Creswells 5 steps to Conduct a
Literature Review (contd)
Step 4: Organize the Literature
Create a file or abstract system
to keep track of what you read.
Each article you read should be
summarized in one page containing
Title (use APA to type the title so that
you can later copy-paste this into the
References section of your paper)
Source: journal article, book, glossary,
etc.
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Step 4: Organize the Literature
(contd)

Research problem: one or two lines will suffice

Research Questions or Hypotheses
Data collection procedure (a description of
sample characteristics can be very handy as
well)
Results or findings of the study
Sort these abstracts into groups of related
topics or areas which can then become the
different sections of your review
Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Creswells 5 steps to Conduct a
Literature Review (contd)

Step 5: Write a Literature Review

Types of Reviews:
Thematic Review: a theme is identified and
studies found under this theme are described.
Major ideas and findings are reported rather
than details.

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
 Step 5: Write a Literature
Review
(contd)
Study-by-study Review: a detailed summary of
each study under a broad theme is provided.
Link summaries (or abstracts) using
transitional sentences. Must be organized and
flow coherently under various subheadings.
Avoid string quotations (i.e., lengthy chunks of
text directly quoted from a source)

Creswell, J.W. (2005) Educational Research: Planning, Conducting, and Evaluating Quantitative and
Qualitative Research
Preliminary Literature
Review
This succinct review of current
literature should:
Provide further contextual
background
Reveal issues related to your study
Describe similar problems in other
organizations
Provide significance to your
approach to the study
Guidelines on Style, Mechanics, and
Language Usage

Does your draft follow the logic or

idea that is presented in your intro
and title?
Avoid overusing direct quotations,
especially long ones
Check style manual for correct use
of citations
(Doe, 2005); Doe (2005); (Doe & Smith,
2005); Doe and Smith (2005); (Black,
2005; Brown, 2006; Yellow, 2007)
Guidelines on Style, Mechanics,
and Language Usage
Avoid using synonyms for recurring
words
This is not creative writing and stay
consistent with terminology
Group I, Phoenix Cohort, Experimental Group
Spell out all acronyms when first using
them
Traditional - American Psychological
Association (APA)
Non-traditional - Collective Efficacy (CE)
Yes - Do NOT use contractions; No
Dont use contractions
Coined terms should be set off by quotes
Guidelines on Style, Mechanics,
and Language Usage

Avoid the following:

Slang cool
Colloquialisms thing >> item or
feature
Idioms rise to the pinnacle >> to
become prominent
Use great care to avoid Plagiarism
What needs to be included in
the Literature review.
Provides contextual background
Reveals related issues
Reviews similar problems elsewhere
Provides significance to your approach
to the study
Includes major/seminar research articles
pertaining to study
Written in an integrated manner
Uses peer-reviewed research
Includes a Reference section
 Writing Your Research
Question(s)

Reflect the problem that the researcher

wants to investigate
Can be formulated based on theories,
past research, previous experience, or the
practical need to make data-driven
decisions in a work environment
 Writing Your Research
Question(s) (contd)

Are vitally important because they, in large

part, dictate what type of statistical
analysis is needed, as well as what type of
research design may be employed
A research question should address only
1 concept
Question must be measurable
 Types of Questions Asked
Once you have identified the topic of study, you
will need to consider the type of question you
want answered and how it will be answered
Two paradigms
Quantitative Paradigm
Generally attempt to quantify variables
of interest. Questions frequently
address how well or how much.
 Types of Questions Asked
Qualitative Paradigm
there are times when we wish to know
not how many or how well, but simply
how. (Shulman, 1988, pg. 7)
Class Exercise
Now youre ready to formulate your own
research question(s)
Sample questions:
Is there a relationship between
participation in an Elluminate chat
session and course grade?
How do 5th grade students
experience the anticipation of
standardized testing?
Research Questions
From Topic to Research Question
A good research topic asks a clear, concise question.
Asking a research question helps you keep a tight
focus on your topic.

Tweaking Your Research Question

A good research topic is broad enough to allow you
to find plenty of material, but narrow enough to fit
within the size and time constraints of your paper.
If your topic is either too broad or too narrow, consider
adding or eliminating the following elements:

Time Period, century, decade, future, Population

Type, age, gender, nationality, species, Geographic
Location country, state, region, Point of View
economic, social, cultural, biological
Assignment 2 Components
(see syllabus for details)

Title Page
Nature of the Problem
Background and Significance of the
Problem
Literature Review
Research Questions
References
Topic ideas

Online chat reference

Types of questions
Subject? Type?
# of turnaways*
Difference in discourse
In-person vs. chat
Partnership studies
Similar libraries with same software
Topic Ideas
E-book usage
Usability studies of
Online tutorial(s)
My Library portals
Analysis of library web sites or
library instruction sites or
pathfinders by best practices
Student learning outcomes in LI
programs
Types of methodologies
QuaLitative Measures
Descriptive
Numbers not the primary focus
Interpretive, ethnographic,
naturalistic

QuaNtitative Measures
N for numbers
Statistical
Quantifiable
QuaLitative measures

Content Analysis
Analyzed course syllabi of
library use through discipline
and level (Rambler)
Studied online tutorials,
applying best practices
recommendations (Tancheva)
 QuaLitative Measures
Discourse Analysis
Analyzed student responses in writing
and discussions to a short film &
compared findings to parallel study
with LIS grad Ss (Vandergrift)
Focus Groups
Discussed how participants
experience & use the library (Von
Seggern & Young)
Studied why students use the
Internet and how much time they use
it (Wilson)
QuaLitative Measures
Interviews
Studied 25 HS students web use for
research assignments (Lorenzen)
Looked at what type of information first
year students need and how they go about
acquiring it (Seamans)
Observation (obtrusive)
Observed students as they conducted
online research & noted their activities
(Dunn)
Observation (Unobtrusive)
Retrieval of discarded cheat sheets to
analyze academic misconduct (Pullen et. al.)
QuaLitative Measures

Think Aloud Protocols

Studied how users navigate a library
web site (Cockrell & Jayne)
Usability testing
Examined students mental models
of online tutorials (Veldof & Beavers)
QuaNtitative measures

CompareThings
Count Things
Survey People About Things
QuaNtitative measures
Comparison studies
Experimental and control groups
Instructional methodologies (Colaric;
Cudiner & Harmon)
Program assessment using before/after
analysis of research papers(Emmons &
Martin)
QuaNtitative measures
Pre & Post Tests (Van Scoyoc)
Measures & Scales
Bosticks Library Anxiety Scale (Onwuegbuzie
& Jiao; Van Scoyoc)
Procrastination Assessment Scale
(Onwuegbuzie & Jiao)
QuaNtitative measures

Numeric Studies
Citation AnalysisBibliometrics
(Dellavalle)
Webometrics (Bar-Ilian)
Ready Made Data Sets
National Survey of Student
Engagement (Whitmire)
College Student Experiences
Questionnaire (Kuh and Gonyea)
The Web
Internet Archive (Ryan, Field &
Olfman)
Electronic journals (Dellavalle)
Library server logs
Common Pitfalls

Problems with population

Sampling?
Representativeness?
Self-selection?
Research Problem #1
A study assessing student learning
outcomes in 2 broad categories
(concepts, techniques) by examining
student research journals in 1 section
of an elective information literacy
course in fall semester.
Research Problem #4
A 2004 article on a library use and
services satisfaction study that used
as its measurement tool a survey
given to every nth person entering
the library building on 40 randomly
selected days throughout the school
year.
Research Problem #5
An outcomes assessment research
project of a 5 year old IL program in
which all incoming freshmen must
participate. Total student population on
campus is divided between 32%
freshmen to senior (or 4 year) and
68% transfer students.
Common Pitfalls

Problems with operationalization

Defining of what is measured
Research Problem #2
An experimental study that proposes a
fund allocation formula for academic
library collections based on the
following:
average of overall book price + average of
overall serial prices * degree level (10 for
undergraduate to 30 for doctorate) / the number of
students enrolled in degree program as majors +
the total number of faculty in the department * three
* total number of students in program.
(OAB + OAS) * D/(Sn +(Fn*3))*Sn
N.B. Not a standard formula
Research Problem #3
A newspaper article you read just the
other day stated that in a recently
published study done at a major U.S.
university, researchers found that
domestic violence affects 1 in every 4
women.
Research Problem #4
A 2004 article on a library use and
services satisfaction study that used
as its measurement tool a survey
given to every nth person entering the
library building on 40 randomly
selected days throughout the school
year.
Research Problem #5
Over a one year period, researchers
studied the occurrence of turn-aways
in a virtual reference service and noted
that the significantly high occurrence of
turn-aways indicates increased need
for virtual reference service.
Common Pitfalls

Problems with generalizability

False conclusions
Transformations
Research Problem #1
A study assessing student learning
outcomes in 2 broad categories
(concepts, techniques) by examining
student research journals in 1 section
of an elective information literacy
course in fall semester.
Research Problem #7
A survey of faculty found that the
majority of those interviewed interacted
most with librarians at the reference
desk. The researchers concluded that
most faculty view librarians in a
servile role.
Keep In Mind That

No study is perfect
All data is dirty is some way or
another; research is what you do
with that dirty data (Manuel)
Measurement involves making
choices
Be Critical About Numbers
(Best 2001)

Every statistic is a way of summarizing

complex information into relatively
simple numbers. (Best)
How did the researchers arrive at these
numbers?
Who produced the numbers and what is
their bias?
How can key terms be defined & in how
many different ways?
Be Critical About Numbers
How was the choice for the
measurement made?
What type of sample was gathered
& how does that affect result?
Is the statistical result interpreted
correctly?
If comparisons are made, are they
appropriate?
Are there competing statistics?
Getting Started

Read to learn; read to analyze

About research methodology
Studies on similar topics
Interesting studies
Non-library studies
Getting Started

Finding a topic neednt be

traumatic
Work projects Research studies
P&T overhaul
Library GO Bond Proposal Project
Library workshop trends
User repair strategies
Getting Started

Data collection involves agreement

& consent
Forge partnerships
At some point you will need to
leave the comfort zone of reading
and literature gathering and
Just get out
and do it!
Questions?
Research methodology
Quantitative Methods
Qualitative procedures
Quantitative Methods

A definition
A survey or experiment that
provides as output a quantitative or
numeric description of some
fraction of the population, called
the sample.
Components of a survey
method

The survey design

The population and
sample
The instrumentation
Variables in the study
Data analysis
The survey design

Purpose of the survey

The research question
Type of survey
Cross sectional
Longitudinal
Form of data collection
The population and sample

Description of the population

Sampling design
Single stage
Multistage
Stratified
Sample selection
The instrumentation
The instrument (tool)
Existing
New
Rating scale
Likert scale: Rating the Items. 1-to-5 rating scale where:
1. = strongly unfavorable to the concept
2. = somewhat unfavorable to the concept
3. = undecided
4. = somewhat favorable to the concept
5. = strongly favorable to the concept

Pilot
Administration
Postal survey
email
Variables and analysis
The research question
Variable in the research
E.g. Number of years of academic study
The questions in the instrument
E.g. How many years of study in a
University
As an undergraduate?
As a postgraduate?
Data analysis
Steps
Bias in the data
Non-response
Statistics, e.g. mean, standard deviation etc.
Components of an
experimental method

Subjects
Instruments and materials
The experimental design
Subjects
Selection
Conveniently
Random (RCT)
Group assignment
Random
Matched. E.g. Ability, Age
Size
Variables
Dependent
Independent
 Randomized Controlled Trial
(RCT)
A true experiment, in which the researcher
randomly assigns some patients to at least
one maneuver (treatment) and other patients
to a placebo, or usual treatment. Key features
= the classic way to evaluate effectiveness of
drugs (or exercise, diet, counseling). Patients
are followed over time (Prospective). If
properly done, an RCT can be used to
determine cause and effect
Instrumentation and
Materials
Description
Validation
Pilot
Content validity
Prediction validity
Materials
The experimental design

Type
Pre-experimental
No control group
Quasi-experimental
Control group, but not randomly
assigned
Single subject design (over time)
Pure experiment
Repeated measures
Change groups
Overview of Qualitative Research
Design

Historical routes in anthropology

Generates new understanding by naming and framing
concepts and themes
Removes bias by questioning preconceived assumptions of
the social group under study
Promotes neutrality through adoption by the researcher of
nave stance or critical discussion, challenges pre-conceived
assumptions of both the researcher and the social group under
study
Produces new understanding about the world, changes the
way power, culture and social interaction are understood
Data Collection in Qualitative Research

Observation (Videoed, non-participant, semi-

participant and participant observation, field notes)
Interviews (individual and group - known as focus
groups, tape recorded and transcribed, field notes)
Secondary data analysis (using written material
collected for purposes other than research)
Questionnaires (unstructured, postal, interviews)
A mixture of all four
Questions in Qualitative Research
In qualitative research questions are open-ended.
Sometimes a check list or topic guide will be used
by the researcher to ensure all the relevant areas
are covered. This is known as semi-structured data
collection. It is used in all four methods of data
collection
Sometimes the only guide is the topic itself and
the researcher collects verbatim or naturally
occurring data. This is known as unstructured data
collection. It is used in all four methods of data
collection
Sampling in Qualitative Research

The sampling method of choice is theoretical

sampling (queuing behaviour)
However, often this is not possible and people
resort to convenience sampling (students) and
snowball sampling (mental health in black and
ethnic minority communities)
Neither of the latter two methods are considered
strong but maybe all that can be achieved.
Research must be viable.
Data Analysis in Qualitative Research
Read and re-read data, become engrossed in it.
Identify themes: common, conflicting, minority
Test themes across the data set, where are they
common, under what circumstances are they found, not
found. This sets the parameters on the interpretation and
generalisation of data
Get more than one person to analyse the data
independently then together
Demonstrate trustworthiness in data analysis
Examples
Biographical continuity
Nursing routines as a method of managing a transient
workforce
Qualitative research

Interpretative research
Process orientated
Researcher(s) are the primary data
collection instrument
Descriptive research
Outputs are an inductive process
References

MSc project web pages

http://www.comp.glam.ac.uk/gis/start.asp?whatfile=gis/gis
rc/msc-proj.htm

Creswell, J. W. (1994) Research

design : qualitative and quantitative
approaches. - Thousand Oaks,
Calif.; London : Sage Publications,
ISBN 0803952546