The following resources are useful to anyone designing a project or activity for an expected outcome. These links provide information on effective methods of implementation and for evaluation and the key factors to be considered.
- Randomised Controlled Trials (RCT) are the best way of determining whether a policy is working. It enables one to compare the effectiveness of a new intervention against what would have happened if you had changed nothing.
example: a randomized experiment was used to test the representation of dark matter as used in planetariums. It found that there was a significant link between the colours used in the visualization of dark matter and the intrepretation by the audience, in this case, school learners. Three groups of randomly chosen learners were exposed to three different representations of dark matter. Representation A, which was used at the planetarium, was found to provide the least effective learning experience.
- Building evidence into education, Ben Goldacre
evidence-based practice in education and myths of randomised trials
excerpt: RCTs eliminate selection bias and can generate a causal conclusion, avoid potentially misleading results from non-experimental work, and provide a quick and digestible conclusion of programme effectiveness
- Role of Evaluation in creating a cyle of project improvement, UK Behavioural Insights Team
excerpt: Using RCTs to improve educational outcomes – an RCT was conducted to determine whether a low cost, in-school remedial education programme could improve school outcomes in India. Women from the local community, who were paid a fraction of a teacherʼs salary, tutored and worked separately with groups of children who were falling behind their peers. Results indicated that the remedial programme significantly improved test scores, particularly in Maths. (page 13)
- Guide to applying behavioural science insights in intervention design, UK Behavioural Insights Team
excerpt: Simplifying the university application process – Process simplifications have been shown to help underrepresented groups go to university, in particular by making it easier for potential applicants from low-income groups to apply for financial assistance. This change involved providing personal assistance to individuals (parents of potential university applicants or the applicants themselves), which included automation and streamlining of parts of the process and testing this against no help and the provision of information alone (with no streamlining or automation). Applicants whose parents received the support were eight percentage points more likely to have attended university in the year following the intervention. In contrast, simply providing information did not increase attendance. (page 14)
- In-depth report on how insights from behavioural science can be translated into practice in project designs, UK Behavioural Insights Team
excerpt: Principles that have the strongest influence on human behaviour and change are embodied in the acronym MINDSPACE (Messenger, Incentives, Norms, Defaults, Salience, Priming, Affect, Commitment and Ego)
- International Initiative for Impact Evaluation (3ie) funds impact evaluations and systematic reviews that generate high quality evidence on what works in development and why. Such evidence on development effectiveness can inform policy and improve the lives of poor people.
The OAD is particularly interested in 3ie and its Evidence Gap Maps (EGM). EGMs consolidate what we know and do not know about ’what works’ in a particular sector or sub-sector. These maps indicate gaps in our knowledge and thus inform us which areas require further research and evidence.
Evidence Gap Map on Primary and Secondary Education
