USING GENERALIZABILITY THEORY IN RELIABILITY ESTIMA-TION OF MEASUREMENTS OF HIGHER-ORDER COGNITIVE SKILLS
ÜST DÜZEY ZİHİNSEL ÖZELLİKLERIN ÖLÇÜMLERİNİN GÜVENİRLİK KESTİRİMİNDE GENELLENEBİLİRLİK KURAMININ KULLANIMI

Author : Nuri DOĞAN -- Özge BIKMAZ BİLGEN
Number of pages : 1-9

Abstract

The purpose of this research is to estimate reliability with Generalizability Theory in assessing high-order cognitive skills of a particular performance. For this purpose, a performance task belonging to science and technology lessons was applied to 50 students and the performances of the students were graded by 10 teachers with a rubric, prepared for the same aim one of which was analytic and the other was holistic. The collected data were analyzed by G-Theory using the EDU-G statistical package program. G study done with students, raters, forms were created by means of variability sources (sxrxf) 2 facet crossed design and D studies were conducted with the created scenarios. According to the G study results, it was seen that the students- who were the objects of measurement- were the greatest variability source, and the other influences stemmed from the raters and forms respectively. Based on the results of D-study, 6 raters and 1 form yielded the optimum results.

Keywords

inter-rater reliability, Generalizability Theory, higher-order cognitive skills

Read: 674

Download: 224