د/ايمان زغلول قاسم

استاذ تكنولوجيا التعليم المساعد بكلية التربية بالزلفي

book B83

The foregoing design principles are of a general nature and may have different
consequences, depending on the level of reporting of the test results and the proposed uses of
the test. According to Ryan (2006), the reporting unit is one of the key characteristics to
consider in designing a score report. In many tests, the results are fed back to the test taker
and other stakeholders (e.g., the teacher and parents). These score reports are aimed at the
individual level. In various testing programmes, the results are also aggregated to a higher
level (e.g., the class or the school) and reported accordingly (Zenisky & Hambleton, 2012).
This has implications for reports; multiple reports are often needed to communicate the results
of one test at various levels.
The importance of the degree in which the report is actionable depends on the
proposed use of the test. Some reports are mainly aimed at communicating a pass/fail decision
corresponding to a summative test purpose, while score reports of tests with a formative
purpose should especially provide suggestions for future learning and/or modifications in
instruction (Bennett, 2011).
6.3. Design Research Approach
This study followed an educational design research approach (McKenney & Reeves,
2012). McKenney and Reeves have distinguished various phases within the design research
process: 1) analysis and exploration, 2) design and construction, and 3) evaluation and
reflection. This process eventually results in two outcomes: An intervention and theoretical
understanding. The process is iterative and flexible, and takes place within the intended
context of the intervention. In this study, several cycles of design and construction were
followed up by (micro-)evaluations and a subsequent redesign.
This study used the approach characterised by McKenney and Reeves (2012) as
research on interventions. The intervention that is the subject of this investigation comprises
the reports generated by the Computer Program LOVS. Thus, the solution to the problem at
hand would be sought in the product form of redesigned score reports. Furthermore, an
Chapter 6
132
existing theory in the form of design principles for score reports was used as the basis for the
design. The results of this investigation aimed to contribute to the theoretical understanding
about score report interpretation.
Various researchers have stressed the importance of field testing and conducting focus
group meetings when (re)designing the contents of score reports (Allalouf, 2007; Hambleton
& Slater, 1997; Hattie, 2009; Trout & Hyde, 2006; Wainer et al., 1999). Hattie proposed using
proficiency tests, focus group meetings, and insights from previous research as sources of
information regarding the adequacy of users‘ interpretation of score reports. He emphasised
the scarcity of empirical studies that have accounted for actual user interpretation. Zenisky
and Hambleton (2012) also highlighted the need for studies that investigate the actual
understanding of score reports. A design research approach is suitable to the problem under
review, because it allows for detailed investigations in the intended context of the intervention
in collaboration with users.
The first author was present in each of the sessions with the focus groups and key
informants. In the focus group meetings, the researcher was accompanied by an educational
adviser. Each session was supported by a PowerPoint presentation (structured based on the
five reports), and was digitally audiotaped, transcribed, and summarised. For the focus group
meetings, the contents of the reports were verified by the contact persons in the schools, i.e.,
member checking (Creswell & Plano Clark, 2007).

الوقت من ذهب

اذكر الله


المصحف الالكتروني