Outline

Literature review
- library training/tutorial

Research design
HIT4010/9010 - Research Methods Ethical conduct of research

Modules 03/04
Reading: Textbook Chapters 4 and 5
Extra reading: Booth, Colomb & Williams (1995)
SUT Guidelines for ethics approval

2 HIT4010/9010, © Swinburne

Literature review Purpose of review

Literature
What other researches have done
Review
To look again (re + view)
A literature review sets out the major themes in the paper, showing
how they relate to previously published results.
e.g., a precise analysis of previous work could explain what methods have
been proposed and tested, how they are similar and where they differ, and
compare their relative strengths and weaknesses.
“Critical” when used to describe reading literature does not mean “criticising”
or “looking for faults”. It comes from the practice of “critiquing” or giving
insightful comments.
Increase your confidence
Provide you with new ideas and approaches
Inform you about other researchers working in this area
Show you how others handling methodological and design issues
Reveal sources of data you may not have considered
Introduce you to measurement tools developed/used
Reveal methods of dealing with similar problems/difficulties
Help you interpret and make sense of your findings

3 HIT4010/9010, © Swinburne 4 HIT4010/9010, © Swinburne

Questions to ask yourself to help

develop your sense of judgement Sources and strategies (1)
Librarians
As you read, actively try to identify contributions and
shortcomings of a paper. Subject liaison librarian
“What are the main results? General reference works
How precise are the claims?
Encyclopaedias: general or specialised
How could the outcomes be used?
What is the evidence?
Text books: understanding a new area
How was the evidence gathered? Index references
How were measurements taken? Related associations (in ICT)
How carefully are the algorithms and experiments described?
ACM
Why is the paper trustworthy?
IEEE (and IEEE Computer Society etc.)
Has the right background literature been discussed?
What would reproduction of the results involve? IFIP
Problem driven
5 HIT4010/9010, © Swinburne 6 HIT4010/9010, © Swinburne

© Swinburne University of Technology 1

 Sources and strategies (2) Using sources (Booth et al. 1995)
Library catalogue
Read critically
Individual University libraries
Coolcat – nation wide catalogue
Evaluate your sources
Identify the sources most valuable to your research problem
Key words, author, title… Skim read many articles and decide which deserve more
Online databases careful attention
Read critical references slowly and carefully
Specialised areas
Provide searches of journal articles and books in a field Take full notes
ACM digital library, IEEE digital library Keep a history of what you have read
Authors, title, journal, volume, page numbers…
Usually provide you with an abstract and more often these Key words
days – a full copy of journal articles
Summary of main points
Citation index Consider using bibliographic package
Avoid World Wide Web (e.g. wikipedia, ‘google’ search engine) e.g. EndNote, ProCite
7 HIT4010/9010, © Swinburne 8 HIT4010/9010, © Swinburne

Taking notes while reading literature (cont) Speedy reading (1) (Booth et al. 1995)
1. Become familiar with geography of source
Books
Notes & Words
Spell technical words and authors’ names correctly from the Read first few sentences in each paragraph of the preface
beginning Look at table of contents for summary chapters
Learn to pronounce the names Skim index for topics with the most page numbers
Learn the right format et al. (not etal) Skim bibliography, noting dates and sources cited most often
Quotes and ideas Articles
In your notes, distinguish between quotes and paraphrases Read the abstract
Keep records of all sources of ideas and also page numbers
Flip through pages to see if there are section headings
for quotes.
Skim the bibliography

9 HIT4010/9010, © Swinburne 10 HIT4010/9010, © Swinburne

Speedy reading (2) (Booth et al. 1995) Library training/tutorial

2. Locate the point of the argument
Hands on introduction to library data bases and
Read the introduction and conclusion electronic journals
Identify the evidence being used to support main claims
3. Identify key-subpoints
Look for places where the writer wraps up a topic (“Finally…”) or
uses transitional works (“First.. Second… Third…”, “Now we EndNote/ProCite training at Swinburne
have to consider…”
Check library website for details
4. Identify key themes
Register for a session
Use subject headings, key points and claims to identify the main
themes
Decide whether these are of relevance to your topic and
deserving of further attention

11 HIT4010/9010, © Swinburne 12 HIT4010/9010, © Swinburne

© Swinburne University of Technology 2

 Discussion items (1)
1. Describe three benefits of conducting a thorough review of existing literature prior to
writing a research proposal.
2. Kirby is conducting a literature review in preparation for his study of “expectations
regarding the sharing of financial and practical responsibilities among married and
cohabiting couples in which both partners are between the ages of 20 and 29”.
Conducting a keyword search on “couples” and “responsibility”, Kirby has generated a
lengthy list of research articles. He decides he will shorten the list of potential articles
Research Design
by eliminate all articles from the list that were not published in prestigious research
journals and include all of the remaining articles in his literature review. What is your
opinion of Kirby’s approach to locating articles to be included in the literature review?
Explain.
3. You are reading a literature review written by a novice researcher in your field. You
notice that the researcher seems to have relied heavily on the Web in preparing the
literature review. A glance at the reference page supports your hunch – a long list of
Internet addresses. What goes through your mind regarding the quality and adequacy
of the literature review in this situation?

13 HIT4010/9010, © Swinburne

Planning Research Design Criteria for research project

Format of research process (diagram) Universality
Research plan could be carried out by any competent person
Replication
Research planning vs Research methodology
Another competent researcher should be able to achieve the
same results as yourself
General criteria for a research project Control
Factors central to the research project must be isolated and
controlled by the researcher
Validation Measurement
Data should be measurable

15 HIT4010/9010, © Swinburne 16 HIT4010/9010, © Swinburne

Nature and Role of Data Admissibility of data

Manifestations of reality Defective data
- Not absolute reality (i.e., Truth) Levels of precision
(Limits of acceptability)
Transient and ever changing Standards
(Specification in research proposal)
Primary data and secondary data What counts as admissible and inadmissible data

17 HIT4010/9010, © Swinburne 18 HIT4010/9010, © Swinburne

© Swinburne University of Technology 3

 Measurement Validity of measurement instruments
Use of existing measurement tools Extent to which measurement tool measures what it is
supposed to measure
Development of customised measurement tools
Face validity
Faulty measurement tool
Looks like its measuring the right thing
Undermine research effort
Content validity
Representative sample of content area
Validity Criterion validity
Reliability Correlation with some other related measure
Construct validity
Measurement of characteristic
that can’t be directly observed
19 HIT4010/9010, © Swinburne 20 HIT4010/9010, © Swinburne

Reliability of measurement instruments Linking Data and Methodology

Extent to which the measurement tool yields consistent Methodology is used to extract meaning from the data
results when the characteristic hasn’t changed Quantitative
Interrater reliability Measuring relationships between variables
Extent to which two or more people rating the same thing Prediction, explanation, control
(traditional, experimental, positivist)
Internal consistency reliability
All items in a tool yield similar results
Qualitative
Complex nature of phenomena
Test-retest reliability
Description and understanding of phenomena from peoples
Extent to which same tool yields same result on two different point of view
occasions (interpretive, constructivist, postpositivist)

21 HIT4010/9010, © Swinburne 22 HIT4010/9010, © Swinburne

Validity of the Method “A Research Checklist” [1]

The accuracy, meaningfulness, and credibility of research
Internal validity (sufficient control)
The extent to which the design and data allow accurate
conclusions about relationships within the data
External validity (generalisation)
Extent to which the results apply to situations beyond the
original (specific) study
[1] Zobel, J. (2004) Writing for Computer Science, Springer Verlag. Pp. 182-183
1. Are the ideas clear and consistent?
2. Is the problem worthy of investigation?
3. Does the project have appropriate scope?
4. What are the specific research questions?
5. Is there a hypothesis?
6. What would disprove the hypothesis? Does it have any
improbable consequences?
7. Are the premises sensible?
8. Has the work been critically questioned? Have you satisfied
yourself that it is sound science?

23 HIT4010/9010, © Swinburne 24 HIT4010/9010, © Swinburne

© Swinburne University of Technology 4

 “A Research Checklist” cont. [1]
9. How are the outcomes to be evaluated? Why are the
chosen methods of evaluation appropriate or reasonable?
10. Are the roles of the participants clear? What are your
responsibilities?
“A Research Checklist” cont. [1]
15. Have milestones, timelines, deadlines been identified?
16. Do the deadlines leave enough room for your advisor to
provide feedback on your drafts, or for your colleagues to
contribute to the material?

11. What activities will others undertake? 17. Has the literature been explored in appropriate depth? Once
the work is largely done – and your perspective has changed –
12. What are the likely weaknesses in your solution? does it need to be explored again?
13. Is there a written research plan? Zobel, J. (2004) Writing for Computer Science, Springer Verlag. Pp. 182-183

14. What forms of evidence are to be used?

[1] Zobel, J. (2004) Writing for Computer Science,Springer Verlag. Pp. 182-183

25 HIT4010/9010, © Swinburne 26 HIT4010/9010, © Swinburne

Ethical Issues in Research Ethics

Protection from harm Ethics: Scientific integrity
Informed consent Professional standards (fabrication, falsification, plagiarism
etc)
Right to privacy
Ethics: responsibility to participants (animal and
Honesty with professional colleagues human)
1946 Nuremberg code
Ethics approval Risk/benefit analysis
Idea of informed consent

27 HIT4010/9010, © Swinburne 28 HIT4010/9010, © Swinburne

Informed consent Ethics approval form

“a person who, in possession of suitable information, Project title
grants authority to someone else to take actions affecting Applicant details
that person” p 22 Beach (1996)
Date of activities
Not just signing a form
Type of activity
Investigator must ensure that the participant understands:
The procedure including potential risks Benefits (to participant & more generally)
Participation is voluntary, and they may withdraw at any time Potential problems
Future use of data
External involvement
External approvals
29 HIT4010/9010, © Swinburne 30 HIT4010/9010, © Swinburne

© Swinburne University of Technology 5

 Discussion items (2)
1. Professor Parsnip wants to know more about the mental process students go
through as they learn about research methods. She also wants to know which
instructional methods are most effective for teaching research methods to her
students. Assume that Professor Parsnip decides to take a qualitative
approach to studying these questions. State a specific research question to
her general research problem that lends itself well to a qualitative research
approach. Then explain why your question is better addressed using a
qualitative, rather than a quantitative, methodology.
2. Swap ‘qualitative’ with ‘quantitative’ above and do it again.
3. Sometimes students believe that taking the time to prepare a complete and
carefully designed research proposal is something to be preached, but not
really practiced. Identify at least two different problems related to research
design that are commonly encountered when doing research. For each
problem, explain how a complete and detailed research proposal would help
the researcher avoid the problem.

31 HIT4010/9010, © Swinburne

© Swinburne University of Technology 6