Innovative Design & Manufacturing Research Centre University of Bath

World-leading research in engineering

design and manufacture.

IdMRC Social Research Methods

Autumn Lecture-Workshop Series

Science
Aim? When is knowledge scientific knowledge?
Criteria?

Knowledge sources? When is research scientific research?

Henri Christiaans

Science
Realism What we observe is real Instrumentalism What we observe doesnt need to be real Social constructivism Theories only get meaning through social and political context

What is Knowledge?
Justified true belief (Platos Theaetetus) The Greeks classify knowledge into 2 types:
Doxa (believed to be true) Episteme (known to be true)

Doxa Epistime
Through Scientific process of inquiry

How do we know what we know?

Define knowledge alternatively Supported by evidence (usually empirical) Conceive knowledge claims in a probabilistic sense Knowledge is a matter of societal acceptance

How is Knowledge Acquired?

Role of science, where science is a convention, related to societal norms, expectations, values, etc. Thus, is science equals any scholarly attempt at acquiring knowledge Science requires conventions to be followed

How is Knowledge Acquired?

Role of science, where science is a convention, related to societal norms, expectations, values, etc. Thus, is science equals any scholarly attempt at acquiring knowledge Science requires conventions to be followed

Knowledge in design
Implicit prioritisation of the (language-based mode of) propositional knowledge (justified true beliefs) seems to exclude certain kinds or formats of knowledge associated with practice, which are often called practical, experiential, personal, or tacit knowledge and which evade verbal articulation.

Knowledge sources
Observation
Experiments/measurements

The Reason
Mathematics/logical reasoning

Intuition Authority (Divine) Revelation

Science based on empirism

Empirism: Knowledge derived from how the world is experienced. Scientific statements are controlled by and derived from our experiences and observations. en

Scientific theories developed and tested by experiments and observations through empirical methods

Questions to be asked
1. 2. 3. 4. Which methods do we plan to use? Which methodology defines the use of methods? Which theoretical perspective do we start from in order to apply the right methodology? Which epistemology feeds this theoretical perspective?

Ontology
1. A systematic account of Existence. Nature of the world around us. 2. (From philosophy) An explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them. 3. The hierarchical structuring of knowledge about things by subcategorising them according to their essential (or at least relevant and/or cognitive) qualities.

Epistemology and ontology

The way of understanding and interpreting how we know what we know. Particular methodologies tend to entail (subscribe to) particular epistemologies and, in their turn, particular forms of ontology

Ontology in Computing Terms

For AI systems, what "exists" is that which can be represented. We can describe the ontology of a program by defining a set of representational terms. Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, an ontology is the statement of a logical theory. A set of agents that share the same ontology will be able to communicate about a domain of discourse without necessarily operating on a globally shared theory. The idea of ontological commitment is based on the Knowledge-Level perspective.

Epistemology
From the Greek words episteme (knowledge) and logos (word/speech) is the branch of philosophy that deals with the nature, origin and scope of knowledge. Refers to our theory of knowledge, in particular, how we acquire knowledge (Hirschheim, 1992).

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretativism symbolic interactionism phenomenology hermeneutics feminism (post)modernism Socialconstructivism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic stat analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal.
Crotty, 1998

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretativism symbolic interactionism phenomenology hermeneutics feminism (post)modernism Socialconstructivism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic stat analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal.

Theoretical perspective
Philosophical point of view which feeds the methodology and offers a context for the process and the logics, and gives our criteria a basis. Cultural differences play a role

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretivism symbolic interactionism phenomenology hermeneutics feminism (post)modernism Socialconstructivism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic statistical analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal. Crotty, 1998

Three Main Epistemologies

Positivist

Interpretivist

Critical

Interpretivism
Interpretivism rests upon idealism: the world is interpreted through the mind; e.g., classificatory schemes of species; the social world cannot be described without investigating how people use language and symbols to construct what social practices; i.e., understand their experience; the social world becomes the creation of the purposeful actions of conscious agents; and no social explanation was complete unless it could adequately describe the role of meanings in human actions Actions are not governed by discrete patterns of cause and effect (as in positivism), but by rules that social actors use to interpret the world

Positivist Science
5 Pillars Unity of scientific method Causal Relationships Empiricism Science and its process is Value-Free Foundation of science is based on logic and maths

Ontology of Positivism
Realism Universe comprised of objectively given, immutable objects and structures, existing as empirical entities, on their own, independent of the observers appreciation of them. Contrasts with relativism or instrumentalism, where reality is a subjective construction of the mind, thus varying with different languages and cultures. While hugely successful in physical sciences, it is not as successful for social science.

Anti-Positivism
Latter part of 19th century Man as an actor could not be studied through the methods of natural sciences that focus on establishing general laws. In the cultural sphere man is free (Burrell and Morgan, 1979)

Post-Positivism
Based on the concept of critical realism, that there is a real world out there independent of our perception of it and that the objective of science is to try and understand it, combined with triangulation, i.e., the recognition that observations and measurements are inherently imperfect and hence the need to measure phenomena in many ways. The post-positivist epistemology regards the acquisition of knowledge as a process that is more than mere deduction. Knowledge is acquired through both deduction and induction.

Simon
Designer
objective Analysis

versus

Schon
Designer

Objective Analysis

subjective Interpretation

design Problem 7-6-2012

design Solution

Design Task
(= problem + situation+ teime)

design Solution

Rational Solving Problem Paradigm

Reflection in Action Paradigm

Rationalist Root

Constructivist Root

POSITIVISM

PHENOMENOLOGY

Methodology
Our strategy and action plans, the design process which defines what specific methods we will choose

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretativism symbolic interactionism phenomenology hermeneutics feminism (post)modernism Socialconstructivism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic statistical analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal. Crotty, 1998

Types of Research
Analytical
Historical Philosophical Literature study Meta-analysis ------------------------------------------------------------------------------------------------------Descriptive Survey (questionnaire, interview) Case study Task analysis Document analysis Correlation anal. Observation Etnographics ------------------------------------------------------------------------------------------------------Explorative Survey Correlational Case study Experimental ------------------------------------------------------------------------------------------------------Experimental Pre-experimental True-experimental Quasi-experimental -------------------------------------------------------------------------------------------------------

Ethnographics

Types of research methods

empirical quantitative inductive participatory

prescriptive

idiographic

nomothetic

descriptive

deductive

unbiased rational

qualitative

Fundamental Research: the Empirical cycle

describing/ interpreting

induction hypotheses

generalising modelling Explaining/ interpreting

knowledge problem

theory
modelling specifying

evaluation
evaluating

deduction prediction

testing

testing
t Hart c.s.

Practice oriented Research: The regulative cycle

describing/ interpreting

diagnosis

generalising modelling designing

problem from practice

plan (problem solving)

deciding evaluating

intervention evaluation
process evaluation
t Hart c.s.

action-process supporting observing

Method
The technique to gather data, related to the research question.

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretativism symbolic interactionism phenomenology hermeneutics feminism (post)modernism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic stat analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal. Crotty, 1998

Qualitative Positivist Research versus Non-Qualitative Positivist Research

QPR Methods Field experiment Non-QPR Methods Math Modeling (analytical modeling)

Lab experiment
Free simulation experiment Experimental simulation Adaptive experiment Field study Opinion research Archival research

Group feedback
Participative research Case study Philosophical research

Table 1. QPR versus Non-QPR Methods (Click on the method for its definition)

35

Type of Research, General Research Approaches, Data Collection Techniques, & Data Analysis Techniques

Design-Led Design-Led

Critical Design
Probes

Expert mindset

User-centered Design
contextual enquiry Usability testing

Design and Emotion

Participatory Design

Lead-user inovation

Dutch/Scandi navian design

Human factors and ergonomics

applied ethnography
Sanders, 2002

Research-Led Research-Led

Participatory mindset

generative tools

Design-Led Design-Led

Critical Design
Probes

Expert mindset

User-centered Design
contextual enquiry Usability testing

Design and Emotion

Participatory Design

Lead-user inovation

Dutch/Scandi navian design

Human factors and ergonomics

applied ethnography
Sanders, 2002

Research-Led Research-Led

Participatory mindset

generative tools

Research background
Epistemology Theoretical perspective
objectivism subjectivism positivism Interpretativism symbolic interactionism phenomenology hermeneutics feminism (post)modernism

Methodology
experimental descriptive survey ethnography heuristic action research discourse anal. evaluation

Methods
scaling questionnaires observation interview focus group case study narratives ethnographic statistic. analysis data reduction cognitive mapping interpretative meth document analysis content analysis conversation anal.
Crotty, 1998

Definitions
Research = the systematic inquiry to the end of gaining new knowledge a researcher = a person who pursues research (e.g., in design). Practice = professional practice (e.g., in design) or to processes usually used in professional practice to produce professional work for any purpose other than the (deliberate) acquisition of knowledge. Practitioner = anyone who works in professional practice.

Design Knowledge

Process (design methodology)

product

people

designers

Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design

Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design. Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.

Design knowledge
Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer. The product dimension asks for forms and materials, and finishes with the embodiment of design attributes: both the intentional world (teleological and functional wishes and needs) in relation with the principal, partial and elementary function and the mans connection with the systemic formal and material part (structure, organization, parts and connections).

Design Research
Scientific Theoretical perspective
Scientific, usually based on physics

Interpretive
Interpretive, focusing on individuals experiences, their construction of understanding, perceptions and interpretation of reality. Often centres on individual creativity and subjective perceptions relating to being creative. Experiences of designers and other design constituents. Tries to identify form of internal creative design activities from observation of externalities. Typically defines design in terms of creativity, art, individual genius and socio-cultural influences

Focus

Empirical realities of the design processes, design objects, design brief and contexts. The core concept of design is defined in terms of these activities.

Design Research
Scientific View of Design
Design is a process. May or may not include creativity. Something, or a specification for something, is created.

Interpretive
Intuitive, involving hidden aspects of human subjective thinking and affective activity.

View of creativity

Human internal activity that results in ideas for new, unusual, highly valued, never before created things, emerging magically from the genius of designers. Creation can be achieved mechanically, by automation Focus on individual creativity attributed to specific designers and or intuitively. socio-cultural influences.

Design Research
Scientific Data collection
Similar to physics and natural sciences.

Interpretive
Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes self reporting data collection. Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes reflective analysis of self reports and self perception. Tacit and embodied skills of designers and users. Culturally-determined knowledge. Embedded meanings.

Analysis methods

Similar to physics and natural sciences.

Knowledge focus

Discipline specific empirical information (along with) elicited representations of tacit information and data that designers use.

Scientific Strengths
1. Techniques to investigate phenomena in ways that are transparent, repeatable, testable, and verifiable. 2. Research methods are expressed in a formal language that enables precise critique of the data collection techniques, methods of analysis, processes that lead to abstractions, and the theory abstractions and conclusions. 3. Correspondence between characteristics of phenomena and the formal defined symbolic language of concepts and operations in which mathematically theories and representations of the phenomena are expressed.

Interpretive
1. Focus on human considerations, such as the human creative aspects of design, and how users and other interpret designed outcomes. 2. Interpretive methods give space for designers and users to explain, in their own words, and from their own perspectives, how they design and use designed outcomes and how they communicate with others about designs. 3. Interpretive methods also allow exploration of opinions of users about cultural aspects of particular designs. 4. The interpretive approach can be extended to draw strength from the use of large data sets by which correlations and measures of confidence in them can be established between individuals stories and the phenomena being studied.

Scientific Weaknesses
Scientific empirical method does not adequately address human subjective, interpretive and experiential phenomena except via physiological substrates.

Interpretive
Main weakness is lack of reliability of individuals evidence, perceptions and interpretations i.e. lack of correlation between what people say and reality.

Evidence of this problem in studies of e.g. witness testimony, reliability of memory, relationships between reported thoughts and physiological evidence, influence of subconscious thinking, mental illusions and delusions in normal people.
False consciousness: peoples representations of themselves are inaccurate or simply wrong. Extends to individuals descriptions of processes, and the social activities that they undertake.

Scientific Contradictions
There is an incompatibility between scientific modelling of design process and inclusion of a process element create a new solution as a subjective human activity. Claims that all sub-fields of design are incommensurate as they use different knowledge (and that the broader field of design is fundamentally fragmented) is at odds with scientific representation of designers working across disciplines and in multi-crossand trans-disciplinary teams.

Interpretive
There is tension between interpretive approaches that focus on experiential subjective phenomenological aspects of human creative design activity and the frequent shift of emphasis onto aspects of design and creative activity that are more accessible empirically using a physical science approach. There is an epistemological inconsistency in claims that Design exists of itself as a phenomena capable of creative agency and action.

Design Research
Loves proposal: a unified basis for design theory bridging these two incompatible approaches. Advantages It provides a coherent epistemological basis for new theories It recasts prior research and theory within a justified integrated framework with a clear epistemology and ontology. This in turn provides the basis for developing a design field.

Foundations for a unified basis

Designs (i.e. the specification for creating or doing something) Designed outcomes (after they are manufactured/actualised) Design activity Design processes The skills of designers The role of design activity Cognitive design processes Behaviour of designers as individuals and in social groups Combinations of the above

Epistemologies Assumptions for Qualitative and Quantitative Research

Deductive logic of quantitative research

Researcher tests or verifies a theory

Researcher tests hypotheses or research questions

Researcher defines and operationalizes variables derived from the theory

Researcher measures or observes variables using an instrument to obtain scores

Creswell, 2003

Inductive logic of qualitative research

Generalizations or theories to past experiences and literature Researcher looks for broad patterns. Generalizations or Theories from Themes or Categories Researcher analyze data to form themes or categories

Researcher asks open-ended questions of participants or records field-notes

Researcher gathers information e.g. interviews, observations

Creswell, 2003

Qualitative vs Quantitative
Purpose Quantitative General Laws Test Hypotheses Predict behavior Outsider-Objective Structured
formal measures probability samples statistical analysis

Qualitative Unique/Individual case Understanding Meanings/Intentions Insider-Subjective Unstructured

open ended measures judgement samples interpretation of data

Perspective Procedures

Qualitative Research
Triangulation By using several data collecting methods field notes, interviews, narratives a complete picture of the phenomenon can be provided

Interpretation: observation of species

a - -

- -

Interpretation
a - ----- - d b - -----

- - c

- - -

- -

-----

- - -

- -

-----

Interpretation: observation of discourse

J (reading) pack is firmly attached to the bike positioning of the backpack was alright fact that the centre of gravity of the backpack is placed rather far to the back of the bike (inaudible) I do we have any em... J there's a problem with potholes .. the backpack tends to slide up and down which adversely influences stability I guess when you hit bumps I isn't that in the negative? J mm yeah well the product was considered ugly well that's solvable (laughter) we can fix that one if nothing else ... it takes a while to get used to cycling with this weight; mistakes are made attaching the fastening device to the bike so it has to be easy to attach K with only one yeah gotta be fool proof so that's part of our J yeah that should be in our spec K functional spec

The role of interpretation

Gap between objects and our representations, in 3 forms ('methodological horrors', Woolgar '88):
1. Indexicality 2. Inconcludability 3. Reflexivity

THANK YOU!