SOCS 2014 IGO Scorecard Introduction
An important feature of the global governance landscape that concerns CIVICUS and its members is how well intergovernmental organisations (IGOs) engage civil society.
Almost all IGOs express a commitment to work with civil society. Many have specific policies and procedures in place to facilitate cooperation with civil society organisations (CSOs).[i] A number of IGOs invite civil society representatives to attend consultations, sit on advisory panels, and take part in monitoring and evaluation. They also have dedicated staff and mechanisms to channel concerns voiced by civil society in their decision-making and programmatic development processes.
Global governance has undergone an incredible transformation over the past 20-30 years. Where once IGOs had to justify the inclusion of CSOs in their work, today it is the exclusion of CSOs that requires justification. From less than 100 CSOs in 1950, today about 3,900 CSOs have consultative status with the UN.[ii]
However, it is not always clear whether commitments to engage civil society are put into practice, or indeed how seriously IGOs take civil society outreach, and how much influence CSOs have in shaping IGO policy and practice. At CIVICUS, we have heard many members complain that they are engaged in a tokenistic basis and that the space offered to civil society remains small, constricted and primarily determined by the IGOs themselves.
The UN’s primary mechanism for civil society accreditation is the Economic and Social Council (ECOSOC). Whilst the extensive list of accredited organisations is impressive, there are many concerns with the UN’s approach on ECOSOC accreditation:
- The requirements for consultative status accreditation are rigorous. CSOs must demonstrate that they adhere to multiple criteria;[iii]
- Access is fragmented. Different CSOs dealing with different UN organs report varying levels of access;
- Accreditation is politicised. Member states can arbitrarily block the application of a CSO;
- After accreditation CSOs generally need a prolonged presence at a UN hub to sustain influence. This is expensive and unfair to CSOs who may not be able to afford dedicated outreach staff;
- The application process for consultative status accreditation can be time consuming, taking anywhere between one to three years.[iv]
A common frustration among our civil society colleagues is that they are rarely invited to play a meaningful part in the most important intergovernmental discussions and are instead relegated to ‘side events’ or to making short (often two minute) statements. Similarly, they note that navigating the numerous accreditation and consultation processes requires a serious commitment of time and effort. And, perhaps most importantly, they worry that the true potential for civil society to be an integral partner to IGOs, and their work remains unfulfilled due to the limited mechanisms for engagement.
“We (civil society activists) are invited to speak for two minutes at these consultations; IGOs listen patiently, but our input never really results in any tangible change.” – CSO representative
Conversely, ‘focal points’ for civil society engagement within IGOs also express concern about how things work. They feel that civil society representatives are not always well prepared to make useful contributions to discussions and that many take an overly combative stance when engaging in IGO consultations, which makes dialogue and cooperation difficult.
“One can’t expect the same level of sophistication from all CSOs – some regularly engage with IGO systems – say in high-level reforms as those who engage with us for the first time…” – IGO focal point
It seems that – whichever side of the coin one looks at and despite the recent improvements in civil society engagement – we are a long way from fully integrating civil society voices into IGO processes. Therefore, as part of CIVICUS’ wider examination of how to democratise global governance in the 2014 State of Civil Society Report, we decided to assess the state of civil society engagement with IGOs.
As a pilot project, we have developed a Scorecard to test, evaluate and improve IGO engagement with civil society. The Scorecard seeks to hold IGOs to their public promises to engage civil society. The aim is not to name and shame. Rather, CIVICUS wishes to work collaboratively with IGOs to help understand how engagement is experienced by civil society actors themselves.
We hope the Scorecard’s multidimensional approach can help IGOs identify the specific areas in which they may be doing well and areas which might require attention. Giving voice to civil society actors is good in its own right, but it will also strengthen the position of civil society advocates within IGOs, who can use the Scorecard to push for greater engagement within their organisations.
This is a pilot exercise. As such, CIVICUS has embarked on this project in order to develop an enduring practice of asking civil society actors about their own experiences. If carried out every year, CSOs and IGOs can track progress over time. But a pilot exercise is a learning event. CIVICUS welcomes all constructive feedback on how the research process may be improved next time.[v]
The remainder of this section outlines the methodology we used in this exercise. Part 2 presents the results of our survey, and Part 3 presents our conclusions and some recommendations for improving IGO/CSO engagement. Part 4 presents a series of profiles of how our selected 10 IGOs are currently engaging with civil society, including some survey results relating to each IGO. Finally, in the Appendix, we present a draft methodology of how survey results could be used to build a Scorecard in the future.
The goal of this pilot phase – and the purpose of this report – is simply to test the feasibility of surveying civil society actors about their engagement with IGOs at the global level. More broadly, the Scorecard intends to offer a starting point for discussion at multiple levels over the role civil society is playing in the international decision-making sphere. With this in mind, we have developed a draft Scorecard methodology, including some survey questionnaires that we believe could be useful in assessing the quality of IGO/CSO engagement.
We developed our methodology through extensive consultations. This included convening several consultations in Geneva, New York and Istanbul with relevant stakeholders. As part of this process, CIVICUS consulted IGO focal points, civil society actors and members of the expert panel. The purpose of these meetings was to present the Scorecard methodology and ask the audience for feedback on the approach and key indicators of assessment. CIVICUS also convened a panel of experts to oversee the development of the Scorecard. The panel members were chosen for their commitment to and experience of facilitating civil society engagement at the global level. Finally, we engaged the services of an academic consultant, to advise us on survey design and implementation.
CIVICUS would like to thank all those who have been involved in supporting this initiative, though it should be noted that the views expressed in this report do not necessarily reflect all the views of all those who have been involved so far.
One critical choice we faced in this pilot phase was choosing which IGOs to review. We settled for an initial ten IGOs (FAO, OHCHR, ILO, UNAIDS, UNDP, UNHCR, UN Women, World Bank, WFP, and WTO)[vi] in this phase based on three criteria:
- IGOs with a global rather than regional focus;[vii]
- IGOs that regularly interact with members of the CIVICUS alliance;[viii]
- IGOs with thematic processes and methods for integrating civil society voices, offering a broad spectrum of organisational strategies and types of CSOs which engage with them.
A second choice was to determine what aspects of engagement we wanted to assess. Based on our initial consultations, we decided to focus on four areas:
- Access: CSO access to the main decision-making body of the IGO. We developed a set of questions to assess how proactively the IGO facilitates civil society engagement within its core decision-making body, as opposed to just at the programmatic level. In doing this we evaluated accreditation mechanisms which have been widely used by IGOs to regulate civil society participation within decision-making structures.
- Policy: Engagement by the IGO with the CSOs in policy dialogue. We developed a set of questions to assess the extent and the stage at which an IGO engages civil society in policy development.
- Programmes: Engagement by the IGO with CSOs in programmatic development. We developed a set of questions to assess whether civil society feels the IGO simply views them as implementers or contractors.
- Empowerment: Empowerment of the CSO by collaborating on relevant IGO initiatives that mattered to the CSO. We developed a set of questions to assess whether the IGO makes an attempt to empower the CSO, for example, by working with the CSO on initiatives that it cares about, beyond programme partnering.
The primary component of our methodology was a survey we developed to ask civil society respondents about their experiences of working with the ten IGOs in the areas listed above. We also developed a survey, which we sent to IGO staff to ask about their experiences of working with CSOs, and conducted interviews.
Once designed and tested, the CSO survey was sent to multiple civil society stakeholders. We sought a purposive[ix] sample of civil society actors who seek engagement with major intergovernmental organisations of interest. A purposive sample was appropriate because the Scorecard wished to survey the experiences of a select group of civil society actors – those engaging with IGOs at the global level – rather than the universe of civil society actors as a whole.
By design, therefore, the survey was not a random sample. Because nonprobability sampling was used, inferences cannot be made on the basis of the survey data on the wider universe of civil society actors. Thus, the Scorecard does not seek – either in design, execution, or analysis – to infer that the opinions revealed by respondent CSOs are representative of any actors beyond these CSOs.
The sample was a broad network of internationally-focused CSOs. It was targeted in two ways. First, CIVICUS’ extensive email list, which contains several thousand contacts, was used as a sampling frame for dissemination of the online questionnaire. Bearing in mind that CIVICUS is a civil society alliance, the vast majority of contacts held by the organisations are civil society actors.[x] Second, an invitation to take the survey was spread through the networks of CIVICUS’ partners, who were asked to forward the link to interested colleagues. The survey was presented as a targeted exercise, to be taken by civil society actors who try to engage with IGOs at the global level.
The online survey was available in English, French, and Spanish, and was open throughout February 2014.
We received 462 responses. 372 (80.2 percent) of 462 were valid responses, meaning the respondent correctly choose one IGO to evaluate. 39 (8.4 percent) choose two IGOs, and the remainder choose three or more.
Some further details of the responses to the survey can be found below.
The survey response profile[xi]
[i] See IGO profiles in Part 4 of this report for examples.
[ii] UN CSO portal, accessed 19/04/2014 at: http://csonet.org .
[iii] Resolutions and decisions of the Economic and Social Council, 1996, accessed 29/04/2014 at: http://esango.un.org/civilsociety/documents/E_1996_31.pdf .
[iv] ECOSOC consultative status and other partnership agreements, accessed 29/04/2014 at: http://www.welcomedesk.org/en/faq/consultative-status-accreditation .
[v] If you have any comments, questions or recommendations regarding the findings or approach outlined in this report please contact .
[vi] The Food and Agricultural Organisation of the United Nations (FAO), Office of the High Commissioner for Human Rights (OHCHR), International Labour Organisation (ILO), United Nations AIDS Programme (UNAIDS), United Nations Development Programme (UNDP), United Nations High Commissioner for Refugees (UNHCR), United Nations Women (UN Women), The World Bank Group (World Bank), the World Food Programme of the United Nations (WFP), and the World Trade Organisation (WTO).
[vii] CIVICUS also decided to assess the civil society outreach arm of an IGO, for example, OHCHR is the primary mechanism for CSO engagement for the UN Human Rights Council, making it unfair to assess the council and more relevant to assess OHCHR. Therefore the term IGO, in this paper refers to both the grouping of sovereign nations and in some cases the civil society engagement arm, which may operate under a different name.
[viii] As part of CIVICUS’ strategic priorities, CIVICUS aims to aid civil society in influencing global processes. CIVICUS’ strategic priorities are available at: http://www.civicus.org/about-us-125/accountability/strategic-directions .
[ix] A purposive sample is a nonprobability sampling method that is used when a highly specific group is being targeted and when the researcher does not want to make inferences about the wider world based on the answers from respondents. A random sample survey would start with a list of all civil society actors (a sampling frame) and then randomly select respondents into the study. Such a list of the universe of civil society actors does not exist, of course. Moreover, because the Scorecard endeavours to investigate a highly specific, niche topic – engagement of civil society at the global level – a purposive sample is appropriate because the Scorecard wishes to investigate a highly specific topic and does not aim to generalise from those targeted to the world of civil society actors.
[x] Because respondents were targeted using CIVICUS’ network of contacts, some of whom passed the survey link on through their own network, a response rate is not available since we cannot define the total number of respondents targeted.
[xi] The graph showing number of answers shows the completion rate for respondents. The low-lying green line, which goes upward around Respondent ID 200, simply shows that approximately 200 respondents provided only a few answers before exiting the survey. Respondent ID is a unique number for each respondent, and this graph sorts respondents according to the number of answers provided. Note that this is not the same as the number of questions answered, since some questions had multiple allowable answers, such as the CSO’s sectoral focus. The graph, therefore, shows that about 300 responses were valid, because they answered most or all questions.