What tools and indicators can be used to measure what has changed and how it has changed?
As part of your monitoring and evaluation plan, it is important to set indicators that are realistic. Often we can be too ambitious when choosing indicators or we choose indicators that may not be relevant to the ‘primary stakeholders’ of the project.
“Increasingly, participatory approaches are helping to close the gap between project implementers and stakeholders and increase dialogue. Working with communities in a participatory and equal way from the outset and then throughout a project is essential to
understanding what change is sought and what change has occurred.” – Equal Access Participatory Monitoring and Evaluation toolkit
An indicator is a quantitative or qualitative metric used to track achievement of an expected result (e.g. number of individuals attending M&E workshops). The indicator can be a number, percentage, ratio, index, etc. An indicator is measured using a tool that allows for collection of the required data. Indicators can tell us if we are meeting our project objectives, what progress we are making and to what extent our targets are being met. They can’t tell us why change is occurring or if our project is having an impact.
In order to help you decide which indicators to choose, a common framework that is used is the ‘SMART’ framework, which encourages us to choose indicators with the following characteristics:
Note: The SMART indicator framework is most suited for quantitative indicators.
The SPICED approach allows for the development of indicators with stakeholders. The idea is that stakeholders can define indicators themselves and therefore use these indicators for themselves to hold the project to account, interpret progress and learn about the changes that the project is delivering. This approach relates more to how indicators should be used:
The SPICED approach is a very useful tool for thinking about how to set participatory indicators. It is qualitative; it appreciates local understandings of change and is a good tool for thinking about why it is important to work with communities. It identifies that different people have different ideas about what change means:
- Subjective: Informants have a special position or experience that gives them unique insights which may yield a very high return on the investigators time. In this sense, what others see as ‘anecdotal’ becomes critical data because of the source’s value.
- Participatory: Objectives and indicators should be developed together with those best placed to assess them. This means involving a project’s ultimate beneficiaries, but it can also mean involving local staff and other stakeholders.
- Interpreted and communicable: Locally defined objectives/indicators may not mean much to other stakeholders, so they often need to be explained.
- Cross-checked and compared: The validity of assessment needs to be cross-checked, by comparing different objectives/indicators and progress, and by using different informants, methods, and researchers.
- Empowering: The process of setting and assessing objectives/indicators should be empowering in itself and allow groups and individuals to reflect critically on their changing situation.
- Diverse and disaggregated: There should be a deliberate effort to seek out different objectives/indicators from a range of groups, especially men and women.