Hype around Impact
These days, there is no question in the development discourse that one of the key words is ‘impact.' The 2005 Paris Declaration and the 2008 Accra Agenda for Action (AAA) speak of managing for results. "We will be judged by the impacts that our collective efforts have on the lives of poor people." (AAA, Paragraph 22)

But what is ‘impact' after all and how do we measure it? We are told by our donors, evaluation consultants and others that ‘impact' is about long term results. ‘Output' refers to an immediate result and there is ‘outcome' as a medium term result that will be achieved when all outputs are laid out. The next in line in the logical order is ‘impact.' You go up the causal chain to reach the highest point which we call ‘impact,' and that is what development is all about.

Make sense? Well, kind of... We all know the world is more complex than that, do we not? And for even more confusion - now ‘impact' is also used by many donors and agencies to refer to any change emanating from an intervention, whether it is short, medium or long term.

So what do the experts have to say? This was the frame of mind when I travelled to Cairo to participate in the conference titled, Perspectives on Impact Evaluation - Approaches to Assessing Development Effectiveness on 29 March to 2 April.

The conference was co-organised by the International Initiative for Impact Evaluation (3ie) [http://www.3ieimpact.org/], the Networks of Networks on Impact Evaluation (NONIE) [http://www.worldbank.org/ieg/nonie/], the African Evaluation Association (AfrEA) [http://www.afrea.org/home/index.cfm] and UNICEF. More than 700 people from many different countries and a wide range of disciplines gathered for this event. Along with donors and CSOs (civil society organisations), there were a number of evaluation experts who were familiar with the language and discourse of the impact evaluation.

Impact is Contested Terrain
It did not take me a long time to find out that the impact evaluation is actually a contested terrain. The dominant paradigm focuses on ‘attribution' (how can you prove that the given intervention triggered and yielded impact?) and ‘counterfactuals' (what would have happened without the intervention?) whereas alternative approaches speak about the importance of making distinctions among simple, complicated and complex contexts and of asking the fundamental question of impact for what. They also introduce and refine tools and methods such as participatory design, constituency feedback and case studies. For the latter, iScale, [http://www.scalingimpact.net/] Keystone [http://www.keystoneaccountability.org/] and others advocate for impact evaluation that suits the nature of development in the 21st century that is anchored in the vision of development owned by people, especially the poor and the marginalised.

Interestingly, this last principle, which is sacrosanct in the work of development NGOs and CSOs, does not find home in the impact evaluation community. It does not mean that there is a shortage of awareness about this principle. Rather, it means that the impact evaluation discourse is often mired in the sea of theoretical contestations and methodological debates - which is very rich, intriguing and academically disciplined - and at the end of the day you can easily feel lost and forget what was the right question you should be asking.

So what does it all mean to civil society practitioners? It means many things, but one of them, I think, is that we should ask back. When asked by your donor, "so what is the impact of your work?," do not rush to find a link between your outputs and larger results at the societal level. Instead, simply ask "what do you mean by impact?" and start a dialogue.

From the conference organisers and participants, I have asked Zenda Ofir, Past President of AfrEA and Priyanthi Fernando, Executive Direcor of Centre for Poverty Analysis (CEPA) in Sri Lanka to share their reflections on the conference.

In Solidarity,
Katsuji Imata