Item discrimination/item analysis

Please ignore all questions in past papers on item analysis and item discrimination. You have not covered this in the course this year GOOD LUCK FOR MONDAY!

RESEARCH DESIGN REVISION LECTURE: MAY/JUNE 2011 PAPER

MAY/JUNE EXAMINATION 2011

SECTION A Research Design – Dr Sumaya Laher & Ms Nicky Israel

Question 1

a)    Quantitative

Tests a hypothesis that the researcher begins with; concepts in the form of distinct variables; measures standardized and systematically created before data collection; data in the form of numbers; focus on objective facts and causal and deductive theory; procedures are standard and replication is possible, analysis conducted using statistics etc…

b)    Protection and welfare of participant; informed consent; use of deception; debriefing; right to withdraw/voluntary nature participation; confidentiality and/or anonymity; differential treatment of the two groups etc…

Question 2

a)    Is a 14-session cognitive behavioural program effective in improving intrinsic mottion?

b)    IV = Cognitive behavioural program
c)    Operationalisation of the IV = participants divided into groups, one group received the intervention, the other did not  
d)    DV = Motivation
e)    Working definition of the DV = “Intrinsic motivation is defined as engaging in activity purely for the pleasure and satisfaction derived from that activity while extrinsic motivation is defined as performing an activity in order to obtain a separate outcome”
f)     Operationalisation of the DV = Sport Motivation Scale (SMS) –
g)    Measurement validity. Any sensible argument re whether the operationalisation of the DV (i.e. use of the SMS) was appropriate for the study or not

Question 3
a)    IV (intervention) manipulated, given to one group and withheld from the other; Experimental and control group, one receiving treatment, one not; Random assignment; True experimental design;Pre-test, post-test control group design
b)    Advantage: causality
Disadvantage: artificiality and generalizability

d)    Advantage: can be administered to large numbers of people, time and cost efficient, provide a type of 'anonymity', yield valuable descriptive information about broad trends
Disadvantage: application from one population / demographic area to another, low rate of questionnaire – return, ambiguous replies, depends on the truthfulness of the subject

 Question 4

a)    Temporal precedence - IV manipulated
Covariation - control and experimental group
Non-spuriousness – random assignment but can never be established completely
Therefore, researcher was justified in making causal conclusions to a high degree
i) Diffusion – could/not impact the study as group attended separate sessions but were from the same community and may have had contact etc…
ii) Selection Bias – could not impact the study as there was random assignment
b)    Testing is a threat because pretest and posttest are the same test or any sensible variation thereof
Can be countered by post-test only or Solomon four groups or parallel versions of the test – any one sensible suggestion
Differential attrition could be a threat because passage indicates individuals dropping out and different numbers dropping out from the two groups or any sensible variation thereof
Can be countered by having sacrifice groups or similar suggestion

Question 5

a)    Sample = Sixty-five students from the University of the Witwatersrand (48 male, 17 female) who enrolled in and completed an introductory cricket course as part of their physical education course
b)    Sampling technique = non-probability, convenience sample, volunteers
c)    Population = characteristics of the sample compared to those of the population  
Ecological = degree to which it is appropriate to generalize from one context to another
d)    Any critical point with reference to ecological or population validity –
e)    Replication, triangulation, choose random sample, more representative sample, larger sample or any other two sensible suggestions

Question 6

a)    Independent t-tests are appropriate because they indicate whether there is a significant difference between the mean scores of two independent groups – in this case the experimental and control group would be independent / any reference to parametric assumptions being met
b)    Statistical conclusion validity
c)    Qualitative paradigm
With any sensible justification – focus groups, subjective, interactive method, data in the form of words, extract themes, greater depth of info

SECTION B Psychometrics – Ms Nicky Israel

Question 1
a)    Embody the knowledge and expertise of others; developed more objectively and scrutinized more carefully than other forms of data collection; represent a collective understanding of a phenomenon; assign numbers to characteristics and allow for the use of statistics; allow for standards of performance to be compared etc…
b)    Closed-ended, Likert-type scale (from Does not correspond at all to Corresponds exactly)
C)    Objective scoring rules: scoring rules guarantee the same outcome
d)    Criterion-based: interpreted by comparison against a set standard

Question 2
a)    Consistency of test scores, extent to which scores affected by random error, poor reliability suggests poor validity etc...
b)    Test-retest reliability
The range indicates moderate to high stability and motivation over three to six months would not necessarily change that much so these estimates would not be unexpected
c)    Internal consistency reliability
Kuder-Richardson 20, Spearman Brown
d)    Internal consistency estimates are affected by the number of items in the scale and the average inter-item correlation. Each subscale measures a different aspect of motivation, and therefore one would not necessarily expect items from the one scale to relate that well to items from the other scale, which would affect the estimate. By calculating internal consistency for each scale, one can assume that all the items in the subscale should relate – 2 marks

Question 3
a)    No, reliability indicates whether the test provides consistent measurement – if the measurement is not consistent, it cannot then assess what it claims or purports to assess (validity) because it has not been shown to assess anything at all
b)    Construct validity – relationship between test scores and measures of other, un/related constructs and behaviours
Convergent validity - expect relationships between scores and scores on the Motivation for Physical Activities Measure-Revised and measures of dedication
Convergent validity – expect relationship between scores and absenteeism from training
c)    Construct validity - relationship between test scores and measures of other, un/related constructs and behaviours
Discriminant validity – do not expect relationship between gender or race and test scores
d)    Criterion-related validity – relationship between test scores and an independent outcome measure
Concurrent validity – assessed in a pre-selected, restricted group  
e)    Content validity/Face validity

Question 4

a)    Bias = systematic error
Fairness= value judgement re decisions/way test used
b)    Fairness – relates to how the test was used
c)    Similar strong correlations with related constructs observed for all groups; similar weak correlations for unrelated constructs observed for all groups; predictions follow same pattern for all groups
d)    Any four sensible points related to bias and the SMS
e)    Any two sensible points related to cultural bias and the exemplar items

Question 6
a)    Inheritance of policies and practices; attitudes to testing; bias; new legislation; multiculturalism/linguilism; resources etc…
b)    Dynamic assessment; computerized/ adaptive testing; interviews etc…

PSYC2005 (RDA IIA) Exam I (Statistics) October/November 2011

PSYC2005 (RDA IIA) Exam I (Statistics) October/November 2011   Question 1 a)      Correlation (use formula for Pearson’s r). The answer is r = + 0.1229 b)      Very weak, positive linear relationship – although yearmark and exam mark move in the same direction, they are only very weakly related.   Question 2 i)        Comparison of two groups (librarians and designers) – no reason to link a specific librarian to a specific designer therefore two independent groups. Assume parametric therefore run a t-test for two independent samples. ii)       Null hypothesis would be H0: Mu 1 = Mu 2 (or Mu p = Mu q) iii)     Alternate hypothesis would be H1: Mu 1 not equal to Mu 2 (or Mu p not equal to Mu 2) because told to test for a difference iv)     Alpha = 0.05 v)      n is 10 for group 1 and 10 for group 2, the degree of freedom is therefore 10 plus 10 minus 2 which is 18.  The test statistic would be t18 = 1.4574. vi)     The critical areas would be 0.025 in each tail; the corresponding critical values from t-tables would be 2.101 and -2.101. vii)   1.4574 is outside the critical areas therefore Fail to Reject H0 at alpha = 0.05. viii)  There is insufficient evidence at the 0.05 level of significance to believe that there is a significant difference between the librarians and designers in mean time to recognize the word.   Question 3 Sampling distribution: Mean IQ (x-bar) is distributed normally with a mean of 100 and a variance of 16-squared over 36 (standard deviation of 16 over the square root of 36). The probability that the average IQ is at least 102 (i.e. greater than 102) is 0.2266 (z =  0.75).   Question 4 i)        Comparison of three groups, parametric therefore ANOVA ii)       Null hypothesis would be H0: Mu 1 equal to Mu 2 = Mu 3 (all the means are equal) iii)     Alternate hypothesis would be H1: at least one pair of means not equal iv)     Alpha equals 0.05 v)      SS error  = 511.46, k = 3, df between = 2, N = 30, df error = 27, df total = 29, MS between = 90.535, MS error = 18.943, test statistic is F (2; 27) = 4.7793 vi)     The critical area would be 0.05 in the right tail; the corresponding critical value from F-tables would be 3.35.  vii)   4.7793 is inside the critical area therefore Reject H0 at alpha = 0.05. viii)  There is sufficient evidence at the 0.05 level of significance to believe that there is a significant difference between the time to complete the task between at least two of the conditions   Question 5 i)        Association between two nominal variables (gender – male/ female and driving speed – below limit, at limit, above limit) therefore Chi-squared test of association  ii)       Null hypothesis would be H0: there is no relationship between gender and driving speed iii)     Alternate hypothesis would be H1: there is a relationship between gender and driving speed  iv)     Alpha equals 0.05 v)      The test statistic would be Chi-squared 2 = 35.796 (please refer to hand out for steps). vi)     The critical area would be 0.05 in the right tail but please use the non-directional row on the table; the corresponding critical value from Chi-squared -tables would be 5.99.  vii)   35.796 is inside the critical area therefore Reject H0 at alpha = 0.05. viii)  There is sufficient evidence at the 0.05 level of significance to believe that there is a relationship between gender and driving speed   Question 6   a)      IV = dominance = nominal; DV = span in centimetres = ratio. Comparison of two linked measurements (groups) therefore matched. DV is at least interval, no other information given re other parametric assumptions (although the sample size is 15 which is too small for Central Limit Theorem to apply and could indicate that the data might not be normally distributed). If assume the other parametric assumptions, including normality, are met, could run a parametric matched pairs t-test; if assume the other assumptions, for example, normality, are not met, would run a Wilcoxan’s MPSR test.  b)      IV = background risk factors composite score – at least ordinal (probably interval); DV = substance abuse score – at least ordinal (probably interval). Assessing relationship therefore use correlation/ regression. c)       IV = TV programme type = nominal; DV = serotonin level  measurement = interval (could argue ratio). Comparison of four groups, DV is at least interval, no other information re parametric assumptions is given. Assuming these are met, could run a parametric ANOVA, if not met could run a Kruskal-Wallis. d)      IV = reading ability = nominal; DV = IQ = interval. One group/sample compared to a population where the standard deviation/ variance for the population is known, therefore use a z-test e)      IV = TV viewership = nominal; DV = annual income = nominal. Relationship between two nominal variables therefore use a Chi-squared test of association.     Question 7 a)      False, zero represents the absence of measurement on a ratio scale. b)      False, a statistic can take on different values and is therefore not fixed. c)       False, the dependent variable in a regression line can be accurately estimated using any value that is within the range of recorded values for the independent variable. Outside this range is extrapolation. d)      True, sample size is n and the standard deviation of the sampling distribution is sigma over the square root of n. As n gets larger, one would divide by a larger number and the answer would get smaller. e)      False, the null hypothesis assumes the status quo i.e. no change and the alternate hypothesis proposes what the experimenter would like to prove.  

REVISION LECTURES - STATISTICS 11 JUNE, RD/PSYCHOMETRICS 14 JUNE

RDA IIA REVISION LECTURE

Statistics

Monday, 11^th June, 11h00-14h00* in SHB 5 (Senate House Basement 5)

*This revision lecture is OPTIONAL. We will try to finish as early as possible, depending on questions and discussion.

NB: Please bring all handouts you have received in tutorials and tables with 

Please prepare the following:

a)         OCTOBER/NOVEMBER EXAM 2011 – PSYC2012: EXAM 1 (Statistics) (pp. 27-31 in Tutorial Pack II)

b)    Any additional examples from any of the exams that you would like to go over for Statistics and/or any questions you have for this section.

If you are not able to attend this revision session, you are welcome to come and consult with the lecturers during their exam consultation times and/or please organise to get notes from a friend.

RDA IIA REVISION LECTURE

Research Design & Psychometrics

Thursday, 14^th June, 9h00-12h00* in SHB 5 (Senate House Basement 5)

*This revision lecture is OPTIONAL. We will try to finish as early as possible, depending on questions and discussion.

Please prepare the following:

a)         MAY/JUNE EXAM 2011 – PSYC2005: EXAM 2 (Research Design & Psychometrics) (pp. 23-27 in Tutorial Pack II)

b)    Any additional examples from any of the exams that you would like to go over for Research Design or Psychometrics and/or any questions you have about either section.

If you are not able to attend this revision session, you are welcome to come and consult with the lecturers during their exam consultation times and/or please organise to get notes from a friend.

Link to access RDA IIA yearmarks

Please click on the link below to access your marks:

https://docs.google.com/spreadsheet/ccc?key=0Am7uz368AQc4dGI0YzlTNEdwVGplUTFCQmZhbExaa2c

RDA YEARMARK ANNOUNCEMENT

Research Design & Analysis IIA (PSYC2005)

Important Notice about Year Marks:

Please note that as a result of a technical error, the year marks posted on the notice board earlier this week were incorrect in a small number of cases. These errors have been corrected, and the year marks now posted should be correct. However, please check your marks carefully, and if you have a good reason to believe that the marks reflected for you are still incorrect, then please contact me as soon as possible!

Thanks,

Kevin

Nicky Israel revised consultation times

Nicky Israel Revised Consultation Times: June Exams and July break   Wednesday 6th June:                           13h00 – 15h00 Thursday 7th June:                               10h45 – 13h00 Tuesday 12th June:                              8h00 – 9h30; 12h30 – 15h00   Thursday 14th June:                            12h30 – 14h00 Friday 15th June:                                  9h00 – 11h00 For short queries, you are welcome to email: Nicky.Israel@wits.ac.za   Please note that except for these times, I will only be available by appointment during the exam period and the break. If I am not in my office during these times, or you need to reach me outside of these times, please email me at: Nicky.Israel@wits.ac.za Alternatively, please contact me on 717-4557 or leave a note with your contact details (preferably including a landline number) either under my door or in my pigeonhole in U211 (Psychology Main Office).  

RDA YEARMARKS

Please note: Yearmarks have been posted on the RDA noticeboard. All queries should be directed to Dr. Kevin Whitehead ( Kevin.whitehead@ wits.ac.za). Some marks are still pending as these were the deferred tests