WEEK 9: Other Sources of Data
• Can be ‘read’, e.g., letters, diaries, autobiographies, websites, photographs, etc.
• If not produced for the purposes of social research they are unobtrusive measures.
▫ Research participants are unaware of being studied and therefore act naturally.
Criteria for Assessing the Quality of Documents Scott (1990)
▫ Genuine and of unquestionable origin
▫ Free from error or distortion
▫ Typical of what it is supposed to represent
▫ Clear and comprehensible
• Diaries, letter, autobiographies
Did the ‘author’ actually write the piece?
Is the report factual?
Does it reflect the true feelings of the author?
Do the documents accurately reflect people of different classes and
May be somewhat elusive
• Visual objects
• Photographs can be
▫ used to illustrate points;
▫ sources of data;
▫ used as prompts to get people to talk.
• Scott (1990) maintains that family photographs are of three types:
▫ Idealization: formally posed
▫ Natural portrayal: candid, not posed
▫ Demystification: revealing, atypical portrayal
▫ The photographs available may not be typical of the event or episode depicted.
▫ What is not photographed may be very significant.
▫ Selection biases (filtering) • What is chosen to be photographed
• What is chosen to be kept
• Social Context of Photographs
• Ask the following questions:
▫ When were they taken?
▫ Where were they taken?
▫ Of whom were they taken?
• Includes census information, voting records, official reports, etc.
• Credibility, representativeness, and meaning are sometimes at issue.
Official Documents from Private Sources
• Includes documents produced by companies and other private organizations.
▫ Biases based on organizational structure (hierarchy) may lead to a distortion of
▫ Access to information requires permission.
▫ To what information is the researcher granted access?
Mass Media Outputs
• Newspapers, magazines, TV, films, etc.
▫ Authorship may be hard to ascertain.
▫ Often the actual focus of the media research
▫ Publications represent particular interest groups and must do so to keep an
▫ Context is important (e.g., culture, humour).
• Includes websites, email, listservs, discussion groups, chat rooms, etc.
▫ Anyone can setup a website so their knowledge claim may not be valid.
▫ The posted information may not be correct.
▫ Wikipedia Secondary Analysis
• Analyzing previously gathered data
▫ Low cost
▫ Less time-consuming
▫ Can provide high-quality data
representative samples, high response rates, national or large-scale
samples, highly experienced researchers with strong control processes
▫ Opportunity for longitudinal analysis
Same survey questions used for several years
▫ Subgroup analysis often possible
Large sample sizes
e.g., comparison of different ethnic or religious groups in the population
▫ Opportunity for cross-cultural (international) analysis
▫ More time for data analysis, since no time is spent gathering data.
▫ The data can be reanalyzed.
Using new quantitative analysis tools
Applying new relationship connections to variables within the data
▫ Fulfills the wider obligations of the social researcher, making full use of data for
which the research participants had to make significant sacrifices.
▫ Lack of familiarity with the data
It takes time to learn the features of a data set.
▫ Complexity of the data:
Data set may contain over one thousand variables and tens of thousands
Data set may be hierarchical.
e.g., contain data at both the household and individual level
▫ The ecological fallacy may become part of the analysis.
This occurs if the unit of analysis for a statement or hypothesis is not the
same as that for the data.
This can be a problem if the data is gathered on a geographic area
(neighbourhood/region) and applied to individuals.
The hypothesis must test the specific variables that exist in a group to
validate any perceptions related to a subset of that group.
e.g., If a region containing a high level of elderly people has a high auto
accident rate, one may be tempted to conclude that elderly people have
high accident rates.
Such conclusions are only warranted if one has data at the individual
▫ No control over data quality Many secondary data sets are of high quality, but some may be of
▫ Absence of key variables
A variable that is of theoretical or methodological interest for the current
project may not be included in the data set or may be measured in
• Includes government data, e.g., Statistics Canada, police statistics, etc.
▫ Often based on populations, not samples.
▫ Reactivity may be less pronounced because participants are not part of a study.
▫ Chance for longitudinal and cross-cultural (international) studies
▫ They record only individuals who have been processed by a government agency.
▫ e.g., crime statistics: much crime goes unrecorded
Perpetrator cautioned instead of charged
Discovered through police surveillance but no actual witness
The ‘dark figure’ in crime statistics
Reliability and Validity
• Reliability can be affected by agency policies
▫ How strictly police enforce the law and how much money is spent on certain
police activities, etc.
e.g., An increased focus on drug crime will increase drug crime statistics.
e.g., Police adjusting reported crime to increase their solve rate
• Reliability of official statistics can be affected by crime variables such as
▫ variations in the likelihood of reporting the crime;
▫ increased police surveillance of particular areas;
▫ variations in police use of discretion;
▫ difficulty identifying and investigating crime related to ethnicity or class;
▫ suicides may not be recognized as such
▫ Official statistics, especially crime statistics, were viewed with suspicion in the
▫ In the 1980s a shift to recognizing the value of official statistics started.
The special problems related to crime statistics should not be generalized
to all official statistics.
▫ Currently official statistics are each evaluated on their own merit. Unobtrusive Measures
• Official statistics may be unobtrusive measures.
▫ People in the study are not aware that they are being observed.
• Other types of unobtrusive measures:
▫ Physical traces, e.g., wear and tear on stairways indicates its level of use.
▫ Archive materials
▫ Simple observation
No control over the behaviour
Simply watch it unfold without getting involved
▫ The use of unobtrusive (non-reactive) methods in conjunction with standard
(reactive) methods (e.g., structured observation)
Structured observation does not fall into these categories because the
observer is known to the observed.
When both are relied upon and the findings compared, the findings
generated by each method may serve to corroborate the other.
WEEK 9: Qualitative Research
• Qualitative research is concerned primarily with words and images rather than numbers.
• It is usually inductive.
▫ The process starts with field research, and then concepts and theories are
• It tends to be interpretivist.
▫ Concerned with finding out what an action or event means to the people
• Often constructionist
▫ Social life is not seen as fixed, but as an outcome of interactions and
• Takes a naturalist perspective
▫ When doing research, the social world should be left as undisturbed as possible.
Kinds of Qualitative Research
• Ethnography/participant observation
▫ Immersed in the social setting
• Qualitative interviewing
▫ In-depth, semi-structured or unstructured
• Focus groups
▫ Interview several people together
• Discourse and conversation analysis
▫ Analyse the language
• Qualitative analysis of texts and documents Steps in Qualitative Research
• Establish a general research question – What interesting social issue are you studying?
▫ e.g., Do high crime communities have low social control?
• Select a relevant site and subjects – Where is the research being conducted and who are
the research subjects?
▫ e.g., Residents in a high crime neighbourhood
• Collect the data – Determine which methods to use. It may be more appropriate to use
▫ e.g., Ethnographic observations and interviews
• Interpret the data – Determine the meanings that the research subject put to activities
that occur in the social environment.
▫ e.g., Residents don’t perceive crime as a problem because there is a low
incidence of violent crime, in spite of the high crime rate.
• Conceptual and theoretical work – Evaluate the data related to your research question
▫ e.g., Crime may be as damaging to a community as outsiders might think if there
are cushioning factors. Although crime is recognized as common, networks of
localized support reduced the potential or at least fear of actual violence. • a) Tighter specification of the research question and
• b) Collection of further data – Here the interpreting and theorizing process is intermixed
data collection. This is the iterative process. Data may confirm or contradict the
interpretation which may lead again to more data collection.
▫ e.g., More interviews with research subjects, perhaps some new or some
• Writing up and findings/conclusions – The researcher must prove the credibility of the
research and why the research matters.
▫ e.g., Provide insight into why crime may flourish in an area essentially under the
nose of residents.
Theory and Concepts
• Qualitative research often involves grounded theory: the use of data to develop
▫ This may involve an iterative process: going back and forth from data to theory,
revising the theory in the process.
• Qualitative research may involve testing theories.
▫ This can be through an iterative process, or occasionally through theory testing
in the conventional sense.
• Blumer (1954): definitive vs. sensitizing concepts
▫ Definitive concepts: defined with nominal and operational definitions, as in
▫ Sensitizing concepts: provide only a general sense of reference and guidance as
to the content of the concept
• Begin with a broad definition of a concept and narrow it down through the research
Reliability and Validity
• LeCompte and Goetz (1982)
▫ Different meanings for key terms
▫ External reliability
Degree to which a study can be replicated
Difficult to achieve in qualitative research
Ethnographer adopts same role as prior researcher
▫ Internal reliability
Do different observers see the same things?
▫ Internal validity
Is there a good match between what is observed and the resulting
Strength of qualitative methods (ethnography)
▫ External validity
Can the findings be generalized across social settings?
Difficult to achieve in qualitative research because of small sample sizes • Alternative Criteria for Evaluation
Do the people studied agree with the interpretation of their thoughts and
actions offered by the researcher?
Conducted through respondent (member) validation
• The people studied may become defensive and try to censor the
• The people studied may not give genuine feedback on what the
researcher produced, but may instead try to please the
• The people studied may not have the expertise to provide
Can the findings be applied to other contexts or people not studied?
‘Thick’ description helps to determine whether transferability is possible.
• Provides enough information to suffice as a database for
comparison to subsequent research findings.
Were proper procedures followed?
Can the study’s theoretical inferences can be justified?
• ‘Auditing’ is sometimes done wherein peers review the research
and procedures to see if the study is dependable.
• May be time consuming and expensive because of the amount of
data gathered in a qualitative research project.
Was the researcher objective and unbiased?
‘Auditing’ can be used to examine this.
Main Goals Of Qualitative Research
▫ Probing ‘beneath the surface’ of social behaviour
▫ Seeing through the eyes of the people studied
In-depth description and emphasis on context
▫ Behaviour that may seem odd or irrational may become more understandable if
the context is described.
▫ Naturalism is an approach that helps the researcher gain an understanding of the
▫ Behavioural observation in its own environment
• Emphasis on process
▫ Showing how events and patterns unfold over time. ▫ A long time spent in the field allows the researcher to understand individual and
social change and its context.
▫ This can also be done with semi-structured interviewing, unstructured
interviewing, and life history approach.
• Flexibility and limited structure
▫ Questions tend to be quite general.
• Especially early in the research
▫ There is usually little or no theory driving the research.
▫ The topics explored in the research may change as the study progresses.
Critiques of Qualitative Research
• It can be too impressionistic and subjective.
• Bias can result from personal relationships that develop during the research.
• It may be unclear as to how a particular topic or theme became the focus of the
• Difficult to replicate
• Reactive effect can be expected.
• Subsequent researchers will bring a different reactive effect.
• Problems of generalization
• However, generalization may not be the goal of the research.
• In-depth description and understanding of meaning may be what the researcher
is aiming for.
• Lack of transparency
• It may be difficult to determine how the research was conducted, why certain
people were chosen for in-depth interviews, etc.
Contrasts between Quantitative and Qualitative Research
• Numbers vs. words
• Point of view of researcher vs. points of view of participants
• Researcher distant vs. researcher close
• Theory and concepts tested in research vs. theory and concepts developed from data
• Structured vs. unstructured
• Generalizable knowledge vs. contextual understanding
• Hard reliable data vs. rich, deep data
• Macro vs. micro
• Behaviour vs. meaning
• Artificial settings vs. natural settings WEEK 10: Content Analysis
• The examination and coding of documents and texts: Printed, visual, aural, or virtual
What gets written/reported?
Where is it written/reported?
Why is this issue written/reported?
When it gets written/reported?
Most importantly, what is left out?
• Can be quantitative or qualitative.
What things can be counted
• Words, including the pairing of certain words
e.g., feminist/spirituality, race/politics, rave/drugs, etc.
• Subjects and themes
This includes both manifest and latent content.
e.g., the subject of ‘women in politics,’ and whether gender stereotypes are invoked or
• Value positions
Positive or negative view
• e.g., the subject of ‘multiculturalism’
What is the presentation format
• e.g. personal ads
• Women present based on appearance.
• Men present based in employment.
• Two key concerns:
Designing a coding schedule
The form where the data are recorded
Designing a coding manual
The set of instructions to coders
Lists what is to be coded, the categories subsumed under each dimension, the numbers
(codes) that correspond to each category, and general guidance for coders
Issues in Devising Coding Schemes
• The categories must be mutually exclusive.
• The categories must be exhaustive.
• The instructions must be clear.
• The unit of analysis must be clear.
• Pilot test will reduce risks of error. Qualitative Content Analysis
• Ethnographic content analysis (Altheide, 1996)
Differs from quantitative content analysis in that the researcher is constantly revising the
themes or categories as the data are examined.
Emphasizes the context in which the documents were generated.
• Semiotics: The ‘science of signs’
• Seeks to expose hidden meanings in texts
• Semiotics Terminology
• Stands for something else
• Thing that indicates an underlying meaning
• The meaning that is pointed to
• The obvious meaning of a signifier
• Secondary meaning that arises with the denotative meaning
• Recognizes that signs may be interpreted differently
The meaning of a text involves the author’s circumstances and the context in which the text
Must be considered during analysis.
Readers: Active or Passive
• Do readers accept the interpretation at face value or do they read with critique and
• Do they re-interpret what is written?
• Social researchers interpret data through their own frame of reference.
• Conclusions reflect interpretations of particular researcher not all possible
The Study of Language
• Conversation analysis
Applies to quantitative and qualitative methods.
Examines how social order is created through communication.
Social order is an achievement not an objective pre-existing phenomenon that we uncover.
Talk is structured.
Talk is forged contextually.
Analysis should be grounded in data. • Conversation analysis
Focuses on reflexivity (responds to context)
A constructionism ontology
Data from naturally occurring situations (naturalist)
Positivist orientation on Replicability
Context means just the words said prior to a response
• Discourse Analysis
Includes elements of conversation analysis
Broader and more flexible
Covers communication other than talk
• Tends toward a constructionist orientation
Strategies employed to create different effects
Advantages of Content Analysis
• In its quantitative form it is very transparent, and therefore easily replicated.
• It allows for longitudinal analysis.
• It is an unobtrusive method.
• It is flexible: It can be used with several kinds of unstructured information.
• Overcomes social barriers to researcher access.
Disadvantages of Content Analysis
• Limitations due to the texts analyzed (e.g., authenticity, credibility, representativeness)
• There is usually some inter-coder and intra-coder unreliability.
• There is a potential for invalid conjecture, especially in discussions of latent meanings.
• It is difficult to answer ‘Why?’ questions using this method.
• Its emphasis on measurement may make it a theoretical in nature.
WEEK 11: Historical – Comparative Research Historical – Comparative Research
• A collection of techniques and approaches.
• Historical time (and often cross-cultural variation) at the centre of research
Object of study is part of the flow of history, in a particular cultural context.
• Comparative Research
Any type of data analysis method
Compares two or more units, such as countries or cultures.
• Comparative research.
Not a separate research technique.
Research perspective or orientation.
• Identifies aspects of social life that are general across cultures as opposed to being
found in one unit alone.
• Concepts and conclusions are less likely to apply only to a specific culture or setting.
• Single culture or setting means a restricted range of possible social activity. Hard to
• Historical social forces that create certain events
• Comparing entire social systems
What is common across societies
What is unique?
• Can strengthen conceptualization and theory building
Generate new concepts.
• New concepts are less likely to be restricted to a single historical time or to a single
Can be grounded in the experiences of people living in specific cultural and historical
Dimensions of Historical Comparative Research
1. Scope: Is the focus on what occurs in one nation, a small set of nations, or many
2. Depth: Is the focus on a single time period in the past, across many years, or a recent
3. Type: Is the analysis based primarily on quantitative or qualitative research?
Historical Comparative Research, Cont’d
• Passage of time is essential to how people construct social reality. Tension between:
Agency (the active, moving, fluid side of people changing social reality), and
Structure (the fixed regularities and patterns that shape social life). • Your own reading of historical evidence is influenced by an awareness of the past and by
living in the present.
• Generalizations and theory are limited.
• Selective facts and limited questions.
• Does not deduce propositions or test hypotheses in order to uncover fixed laws.
• No replication since each researcher has a unique perspective and assembles a unique
body of evidence.
• Does offer plausible accounts of societies.
• Linking the micro to the macro.
• Comparative researchers compare across cultural-geographic units.
• Historical researchers investigate past contexts (usually in one culture) for sequence and
• Can combine both to investigate multiple cultural contexts in one or more historical
Every period or society has its unique causal processes, meaning systems, and social
Societies and time periods may not be equivalent.
Tension between the concrete specifics on a context and the abstract ideas a research uses
to make links across contexts.
Comparative research: The specifics of the societies are translated into a common,
Historical research: Theoretical concepts are applied across time.
Steps in historical comparative research
1. Conceptualizing the Object of Inquiry
2. Locating evidence
3. Evaluating Quality of Evidence
1. How relevant is the evidence to emerging research questions and evolving
2. How accurate and strong is the evidence?
4. Organizing Evidence
6. Writing a Report
An Event A record of the past
Method of doing research
• Primary Sources
• Secondary Sources
• Running Records
• Oral History Research Method
• Why Collect Oral Histories? (An Example of an Oral History)
Limitations of primary sources
• What has survived is not a non-random sample.
• Where are they? What is their condition?
Limitations of secondary sources
• Cannot be used to test hypotheses
• Are not ‘value-free’
• Selection of events
• Not linked to theory
• Narrow focus on actions of specific people
• Organization Equivalence
• Lexicon equivalence
• Contextual equivalence
• Conceptual equivalence
• Measurement equivalence
• On your index card, please answer the following questions:
1) Of everything that has been covered in class so far, what do you least want to see on the
i.e. What are you confused about most and would like us to spend more of our
time in class on?
2) Is there something to do with research methods that we have not (or will not) cover that
you would like to talk about?
WEEK 11: Indigenous Ways of Knowing
Learning Objectives (Frideres, 2011, p. 41)
1. To understand the philosophical underpinnings of knowledge.
2. To be able to compare axioms / postulates for both Indigenous and Western ways of
3. To gain a basic understanding of the Indigenous paradigm for knowing and its relation to
Western ways of knowing.
Indigenous Population in Canada
• In 2011 the National Household Survey (NHS) showed that Aboriginal people represent
4.3% of the total Canadian population. Aboriginal people accounted for 3.8% of the
population enumerated in the 2006 Census, 3.3% in the 2001 Census and 2.8% in the
• The largest numbers of Aboriginal people lived in Ontario and the western provinces
(Manitoba, Saskatchewan, Alberta, and British Columbia). Aboriginal people made up
the largest shares of the population of Nunavut and the Northwest Territories.
Epistemological Questions, or How We Know What We Know
The theory of the nature of existence, or the nature of reality. Is there one “real” world that
each of us observes differently through our own senses, or do various worlds exist, depending
upon the point of view of the observer? There is no way to come to a definite answer to this
question, so people develop an ontological set of beliefs and take it on faith from there. Once a set of beliefs is established regarding what is “real,” research then follows these beliefs in an
attempt to discover more about this agreed upon reality. Ontology is thus asking, “What is
• Ontology – what is your belief in the nature of reality?
• Epistemology – how you think about reality? How do we come to know what is true?
• Methodology - Refers to the theory of how knowledge is gained, or in other words the
science of finding things out. Your view of what reality is, and how you know this
reality, will impact on the ways that more kno