By Matthew Stephenson
While the anticorruption field is rife with disagreements and debates about “what works,” one thing that pretty much everyone can agree on is the need for more and better evidence. This is why it is so important that governments and other organizations engage in appropriate monitoring and evaluation (M&E) activities to assess the impact of their anticorruption work. Lots of organizations conduct M&E activity—but how good is it? A new report developed at the Centre for the Study of Corruption and published with the U4 Anticorruption Resource Centre seeks to provide a comprehensive review of anticorruption M&E in development cooperation. The report, which is based on a structured review of 91 evaluation reports published by 11 development agencies and non-governmental organizations, examines the M&E evidence available for a range of anticorruption measures implemented in a wide range of countries.
The findings are disappointing. Although there are some high-quality evaluations, the review demonstrates there are systematic problems with the quality of the evidence produced through M&E.
Part of the challenge lies with how many anticorruption programmes are set up. Most evaluations are not based on structured research designs, and lack baseline studies, monitoring information, and appropriate outcome indicators. Additionally, over half of the evaluations reviewed do not incorporate any contextual analysis. Advancements in corruption measurement have also rarely been applied in practice in evaluations.
The findings, though disappointing, are not surprising—at least not to those familiar with how these evaluations are typically conducted. The median timeframe given to external consultants to complete their M&E work is three months, which is not a lot of time to understand a complex program and produce independent analysis. Evaluation also usually only starts at the end of a program’s lifespan. M&E is often viewed as a compliance requirement, rather than something the organization really values, and as a result many evaluation reports are formulaic and insufficiently analytical. Worse, development agencies and NGOs under great pressure to show “results” and “success” may be reluctant to undertake the sort of serious, critical assessment of their projects.
To be sure, evaluating anticorruption efforts is very challenging. Change is hard to observe and measure, and various complex factors (political, economic socio-cultural, and others) play a part in determining success. Nevertheless, while M&E is inevitably imperfect, this is a huge missed opportunity. There are better approaches to anticorruption M&E that organizations can and should adopt (see, for example to here and here). For starters, organizations ought to review the timing of evaluative work, draw on some of the innovative research designs developed in the evaluation field, and make use of more appropriate indicators and measurement tools. And speaking more broadly, academics and practitioners should engage more seriously with M&E, and question whether current approaches are delivering the quality and types of evidence needed in this field.
