INSTRUMENTATION
- Instruments used in research must be
reliable, valid and suitable to answering the question.
- Invalid instruments cannot provide
valid measurements of the variable of interest and instruments do not always
have the same validity in different contexts or over time.
- Instruments may also interfere with
what they are attempting to measure i.e. react with the phenomenon they are
attempting to measure (instrument reactivity) e.g. A subject becoming upset due
to traumatic memories while filling out a questionnaire on PTSD
- use
valid, reliable, fair instruments
HISTORY
- Refers to environmental events that
are unrelated to the variables of the study which occur during the research and
may influence the results
- E.g. – class attendance and the
weather; political situation and media exposure; fuel consumption prices and
traffic safety
- random
assignment, matching (personal history)
MATURATION
- Refers to any systematic changes/
processes in an organism’s biological or psychological condition over time
which are not related to or produced by the study i.e. any physical,
psychological or emotional changes occurring in subjects over time
- Particularly problematic in studies
involving children or longitudinal studies.
- E.g. – a two-year case study of a
college graduate entering a new job
- random
assignment, shorter duration
TESTING
- Refers to the possible effects of
having already taken a test on an individual’s score when s/he takes the test a
second time i.e. gains in performance on test-retest designs may be at least
partially due to testing effects, as opposed to the IV.
- Problems of familiarity with format,
administration and content
- E.g. – taking the same final year exam
twice
- no
pretest, alternate forms of the same test
STATISTICAL
REGRESSION
- In studies involving repeated testing
under the same conditions, there is often a trend for extreme scores in a distribution
to move or regress towards the mean
- no
pretest, alternate forms of the same test, random assignment (Solomon Four
design allowing influence of statistical regression to be estimated)
SELECTION
BIAS
- Occurs when there is a breakdown of
random assignment i.e. there is bias (a systematic difference) in the division
of the EG and CG
- NB: selection bias applies to
selecting the EG and CG from the sample – it does not apply to selecting the
sample itself (issues of bias in selecting the sample from the population apply
to threats to population validity – please refer to Lecture 8)
- Occurs when one group has particular
characteristics that the other does not i.e. non-equivalent groups.
- random
assignment, matching
DIFFERENTIAL
ATTRITION
- Attrition refers to the loss of
subjects from the study, often caused by withdrawal (refusal to continue) or
life events
- Differential attrition occurs when the
numbers of subjects lost are not equal between the groups i.e. differential
subject loss.
- Problematic because it may not be
random, thus introducing an extraneous variable, and possibly limiting the
generalisability of results
- quasi-controls,
motivational manipulations
DIFFUSION
- The spread of treatment effects from
the EG to the CG
- May contaminate or confound the
effects of the treatment
- Placebo
control method, keep groups apart, shorter duration, deception, disguised experiment
o
COMPENSATORY EQUALISATION
§
Untreated
groups or subjects (CG) learn of the treatment received by the EG and demand
the same treatment, thus confounding the effects of the treatment
§
Members
of the CG may also attempt to compensate on their own, thus confounding the
effects of the treatment
§
E.g.
a new teaching method
o
COMPENSATORY RIVALRY (The John Henry
Effect)
§
Untreated
groups or subjects (CG) learn of the treatment received by the EG and thus work
extra hard to compensate and exceed the EG i.e. the superiority of the
EG is not demonstrated
o
RESENTFUL DEMORALISATION
§
Opposite
effect of compensatory rivalry, and occurs when untreated subjects or groups
(CG) learn of the treatment received by the EG and thus become less productive,
efficient or motivated as a result
§
Can
create appearance of benefit from the treatment, even when this is not the case
HAWTHORNE
EFFECTS
- Distortions in behaviour that occur
when people are aware that they are being observed/participating in a study
- unobtrusive
studies/naturalistic observation (ethics?), adaptation period, deception
GOOD
SUBJECT / HALO EFFECT / FAKING GOOD
- Refers to the possibility that subject
responses may be distorted by a desire/need to meet social expectations/ please
the researcher/appear better than reality
- deception, disguised experiment,
sacrifice group, motivational manipulation, bogus pipeline strategy
FAKING
BAD
- Refers to the possibility that subject
responses may be distorted by a desire to thwart the researcher/appear worse
than reality
- Opposite of halo effects/faking good
- deception, disguised experiment,
sacrifice group, motivational manipulation, bogus pipeline strategy
EXPERIMENTER
EFFECTS
2 ways in which experimenter can
influence study, viz.
o NON – INTERACTIONAL EXPERIMENTER
EFFECTS
§ Bias
(systematic errors in interpretation/observation)
§ Dishonesty
(intentional effect)
§ Sloppiness
(intentional effect)
§ ask an objective colleague/expert, professional
review panels
o INTERACTIONAL EXPERIMENTER EFFECTS
§ Biosocial
& psychosocial effects (e.g. gender)
§ Experimenter
expectancy/ self-fulfilling prophecy
§ quasi-controls, blind techniques, automation
