TY - JOUR
T1 - An exploration into the criteria used in assessing design activities with adaptive comparative judgment in technology education
AU - Buckley, Jeffrey
AU - Canty, Donal
AU - Seery, Niall
N1 - Publisher Copyright:
© 2020 Educational Studies Association of Ireland.
PY - 2022
Y1 - 2022
N2 - The use of design assignments for teaching, learning, and assessment is considered a signature of technology education. However, there are difficulties in the valid and reliable assessment of features of quality within designerly outputs. In light of recent educational reforms in Ireland, which see the introduction of classroom-based assessments centring on design in the technology subjects, it is paramount that the implementation of design assessment is critically considered. An exploratory study was conducted with a first year cohort of initial technology teacher education students (N = 126) which involved them completing a design assignment and subsequent assessment process through the use of adaptive comparative judgement (ACJ). In considering the use of ACJ as a potential tool for design assessment at post-primary level, data analysis focused on criteria used for assessment. Results indicate that quantitative variables, i.e. the amount of work done, can significantly predict performance (R2 =.333, p <.001), however qualitative findings suggest that quantity may simply align with quality. Further results illustrate a significant yet practically meaningless bias may exist in the judgement of work through ACJ (ϕ =.082, p <.01) and that there was need to use varying criteria in the assessment of design outputs.
AB - The use of design assignments for teaching, learning, and assessment is considered a signature of technology education. However, there are difficulties in the valid and reliable assessment of features of quality within designerly outputs. In light of recent educational reforms in Ireland, which see the introduction of classroom-based assessments centring on design in the technology subjects, it is paramount that the implementation of design assessment is critically considered. An exploratory study was conducted with a first year cohort of initial technology teacher education students (N = 126) which involved them completing a design assignment and subsequent assessment process through the use of adaptive comparative judgement (ACJ). In considering the use of ACJ as a potential tool for design assessment at post-primary level, data analysis focused on criteria used for assessment. Results indicate that quantitative variables, i.e. the amount of work done, can significantly predict performance (R2 =.333, p <.001), however qualitative findings suggest that quantity may simply align with quality. Further results illustrate a significant yet practically meaningless bias may exist in the judgement of work through ACJ (ϕ =.082, p <.01) and that there was need to use varying criteria in the assessment of design outputs.
KW - adaptive comparative judgement
KW - assessment
KW - design
KW - educational reform
KW - technology education
UR - http://www.scopus.com/inward/record.url?scp=85090763591&partnerID=8YFLogxK
U2 - 10.1080/03323315.2020.1814838
DO - 10.1080/03323315.2020.1814838
M3 - Article
AN - SCOPUS:85090763591
SN - 0332-3315
VL - 41
SP - 313
EP - 331
JO - Irish Educational Studies
JF - Irish Educational Studies
IS - 2
ER -