easily published than studies with significant results. If that is the case, this meta-analysis
would not cover the complete population of studies and would be potentially biased. This
issue was addressed with the fail-safe N method (Orwin, 1983), which estimates how many
studies reporting non-significant effects sizes would be needed in order to get a specific result
combined with the studies in this meta-analysis.
In order to assess if effects sizes of studies with different characteristics differ in their
mean effect sizes, we used analysis of variance for effect sizes. To establish the relationships
between effect sizes and several study characteristics simultaneously, we performed a
weighted regression analysis. All analyses were carried out using adjusted SPSS macros
written by Wilson (version 2005.05.23).
Effects of Feedback in a Computer-Based Learning Environment on Students‘ Learning Outcomes: A
4.3.1 Results of the Selection Process
In the primary literature search conducted in May 2011, 2,318 publications were
retrieved. After removing duplicate publications, 1,609 unique publications remained. These
publications were subsequently subjected to the selection criteria. The results of the selection
process can be found in Table 4.1.
In total, 125 studies retrieved in the primary search appeared to match the selection
criteria based on their titles and abstracts. However, some of the selected studies (n = 86)
were excluded after retrieving their full text versions because they did not report sufficient
information, did not contain a sufficient sample size, or did not compare different types of
(standardized, response-based) feedback. Other studies were excluded because their full text
versions could not be obtained (n = 3). For two studies, both a dissertation and journal article
were included in the selection. For these cases, the dissertations were excluded from the
selection. The primary search resulted in 36 relevant studies for this meta-analysis.
Using the ancestry approach (White, 1994), another 18 studies were retrieved, and 4
met the inclusion criteria. Six of the studies that had not yet been retrieved in the primary and
secondary searches and were included in an existing review study appeared to be relevant.
However, none of these studies were eligible for inclusion. Thus, the secondary and tertiary
searches resulted in another four eligible studies, which produced a total of 40 studies for this
meta-analysis. From these 40 studies, 70 effect sizes were obtained. The number of effect
sizes will be indicated by k in the remainder of this article.
4.3.2 Characteristics of the Selected Studies
The studies in this meta-analysis were published between 1968 and 2012. Of the 40
studies, 30 appeared as a journal article and 10 were published in the form of a doctoral
dissertation. The majority of studies were conducted at a university, college, or in adult
education (n = 32, k = 56). The number of studies that have been carried out in secondary
education was six (k = 12), and two studies were conducted in primary education (k = 2). The
studies were categorised based on their subject or content area. Four categories were
distinguished: 1) social sciences (n = 5, k = 9), e.g., psychology and education, 2)
mathematics (n = 6, k = 8), e.g., introductory statistics and algebra, 3) science, biology, and
geography (n = 18, k = 29), e.g., chemistry, climate change, and medical education, and 4)
languages (n = 13, k = 24), e.g., vocabulary and Spanish as a second language. The majority