Aid under the microscope: Scrutinising DFID’s performance

On Thursday 18 September 2014, ARI hosted a public meeting focusing on the Independent Commission for Aid Impact (ICAI). A presentation on ICAI’s work and remit by Chief Commissioner Graham Ward CBE was followed by questions from the audience. Marcus Cox, a lead evaluator for ICAI, and Alexandra Cran-McGreehin, head of the ICAI secretariat, joined Graham Ward as respondents.

Timely appraisal

The meeting took place shortly after:

the parliamentary International Development Committee (IDC) released its annual review of ICAI’s performance in which the IDC emphasised the need to safeguard the commission’s independence and accountability as it completes its first four year mandate and prepares for the next, commencing in May 2015
the evidence session at which the IDC scrutinised ICAI’s reports on “How DFID Learns” and DFID’s private sector development work
the second reading of the Aid Bill, which would enshrine in law a commitment to spend 0.7% of the UK’s national income on overseas development assistance (ODA)

ICAI’s mandate: impact, effectiveness and value for money

ICAI publishes reports with the aim of maximising “the impact and effectiveness of the UK aid budget for intended beneficiaries and the delivery of value for money for the UK taxpayer.” Since its inception in 2011, ICAI has issued 35 reports, 24 of which have rated the aid programme under scrutiny as performing “well” (green rating) or “relatively well” (green-amber rating).

The Chief Commissioner stressed that the body is independent from government and reports to Parliament rather than ministers. ICAI has no mandate to comment on policy. Rather, it assesses how effectively DFID and other ODA-spending departments implement policies; and it issues recommendations to which the relevant ministry is obliged to respond within three weeks.

In 2014-15 ICAI is due to publish reviews on:

the impact of DFID scaling-up its work in fragile states;
funding for multilateral donors;
security and justice programming;
the UK’s International Climate Fund.

ICAI will also revisit anti-corruption measures and private-sector development, both covered in previous reports.

ICAI’s impact: convincing DFID to take remedial action

Graham Ward highlighted two of the ICAI reports which had had a significant impact on DFID programming, implementation or oversight. The review of TradeMark Southern Africa (TMSA) prompted ICAI to issue its first ‘red’ rating and the Secretary of State for International Development, Justine Greening MP, closed TMSA in December 2013. Two days ahead of ICAI releasing its damning report, DFID acknowledged critical shortcomings in the programme’s governance; financial management; procurement; transparency of spending; value for money, delivery, and impact.

DFID’s education programme in Nigeria was also the subject of a critical ICAI report. When an immediate response to the report was not forthcoming, ICAI’s concern mounted. Although ICAI voiced concerns about the UNICEF-managed Girls’ Education Programme (GEP) in November 2012, DFID did not take any action until October 2013. ICAI continued to articulate its concerns to the IDC and Secretary of State, advocating for the funds allocated to this programme to be channelled into the better-performing Education Sector Support Programme in Nigeria (ESSPIN). Graham Ward announced that he had received a letter from DFID that very afternoon containing a commitment to redesign the programme.

Questions from the floor

Graham Ward, Marcus Cox and Alexandra Cran-McGreehin answered questions from the floor for more than an hour. These included:

Q. Gideon Rabinowitz (ODI): “How does ICAI’s methodology differ from DFID research, evaluations by academics, or reports by parliamentary committees?”

A. “ICAI works to a different timetable, and focuses on taking a snapshot assessment of a programme, rather than evaluating it at conclusion. This enables ICAI to ask a wider range of questions, triangulate information and test the claimed impact on programme beneficiaries.

Q. Andrew Weir (Africa Confidential): “Is ICAI able to investigate issues of contemporary concern, such as the abuse of budget support?”

A. “ICAI remains sufficiently independent and nimble to respond promptly, as demonstrated by the rapid review of DFID’s humanitarian response to Typhoon Haiyan in the Philippines – vividly described by ICAI Commissioner Mark Foster in his blog. Budget support was a major focus of ICAI’s during its first year of operation, culminating in a green-amber report, issued under the direction of Commissioner John Githongo.

Q. Kirsty Newman (Evidence into Action Team at DFID): “Could ICAI do more to scrutinise DFID regarding the extent to which the rigorous evidence base is taken into account by project planners and those implementing and adapting programmes?”

A: “ICAI will review the way in which impact is understood and programmes are optimised to achieve either short- or long-term results, or a balance between them. ICAI will also be tracing the use of different results-management tools by the department. Holding DFID to account on its use of evidence is something which ICAI has increasingly focused on, particularly in business cases, where there has been a degree of cherry-picking to justify the programme rather than reflecting on what has worked elsewhere.”

Q. Julius Mbaluto (Informer East Africa): “Despite ICAI’s work, I continue to sense a disconnect between DFID and beneficiaries. What are you doing to address that?”

A. “Engagement with beneficiaries has been a primary focus of ICAI’s work since the very beginning, and we continue to strive for their views to be reflected in our reports. ICAI shares your concern about a disconnect between DFID and its beneficiaries and we have pushed DFID quite hard on this. Unfortunately, the machinery of delivering aid has become heavier and heavier over recent years, with the result that the time available to engage with beneficiaries has been curtailed. However, as the result of our work to promote beneficiary feedback mechanisms have become integrated into the M&E frameworks.”

Q. Mike Woodridge (BBC): “How much do you cost?”

A. “ICAI spent £3.5million in 2013, mostly on contractors. The Chief Commissioner is paid £600/day and the other Commissioners receive £300/day. Full details of expenditure in 2013/14 can be found on pages 54-55 of the ICAI Annual Report. By comparison, DFID sets aside £150m per annum for its own internal evaluations, illustrating that ICAI plays a unique and positive role for a relatively small sum.

Q. Nicola McIvor (Oxfam): “In your evaluations have you found that DFID’s quest to achieve value for money has the unintended consequence of making DFID more averse to taking risks, working with smaller organisations, or committing to extended timeframes?

A: ICAI has announced publicly and privately that we are not averse to risk-taking by DFID. Indeed, we believe that innovation can provide real value for the poor. ICAI’s annual report comments on some of these themes (from page 26).