SOCS 2014 IGO Scorecard Results

Key Findings

• • • Obstacles: The three most commonly identified obstacles were member states overriding CSO voices, consultations that had no outcomes and weakness in the outreach mechanisms of IGOs.
    • Priorities: The three priorities that ranked highest were greater focus on local or regional outreach, greater focus on identifying appropriate interlocutors to reach different types of CSOs and decentralised CSO outreach strategies.
    • Access: CSOs reported that IGOs were overly selective in choosing whom they sought to engage, not proactive enough in reaching out to civil society and provided weak access to the main decision-making body of IGOs.
    • Influence on Policy: CSOs reported not feeling listened to on policy issues, and a major obstacle identified was organisation of dialogues without tangible outcomes.
    • Programmatic Delivery: A slight majority of CSOs felt that IGOs were only interested in them for their ability to deliver programmes and projects, though a large minority did not strongly report this complaint.
    • Empowerment: CSOs were quite split on the extent to which IGOs actively sought to strengthen them and collaborate with them on initiatives that matter to CSOs. This speaks to different experiences across various IGOs. Some CSOs have had positive experiences at some IGOs, others much less so.

Improving IGO outreach mechanisms

We asked CSOs what IGOs should prioritise to improve civil society access to their main decision-making body. Respondents ranked eight items in order of priority, where 1 was the highest priority. Across all IGOs, the highest priority was greater IGO focus on local or regional outreach to civil society. The average ranking for this priority was 2.9. It was a high priority for both accredited (3.1) and non-accredited (2.9) civil society actors. Taken together, the top three priorities ­– focus on local/regional outreach, reaching out to different CSOs and decentralised outreach strategies – reinforce the idea that IGOs should do more to reach beyond the ‘usual suspects’ and engage a wider range of civil society. Interestingly, outreach in different languages, improvements to accreditation processes and increased capacity for IGO civil society focal points all emerged as low priorities.

We also asked IGOs what issues they would prioritise for improving CSO engagement within their organisation. The table below highlights responses from IGOs.[i]

Civil society priorities for improved access to IGO

Priorities of IGOs for improving access to their decision-making body

The above tables suggest a difference in the priorities for engagement between CSOs and those who work within IGOs. Some useful findings emerge from the data within which there are common themes. Both CSOs and IGOs placed greater local or regional outreach in the top three priorities for engagement, highlighting a well-established idea that local solutions should be found for local issues. IGO staff also prominently placed “increased capacity for CSOs to engage with their systems” as a key priority. This highlights that from an IGO perspective, CSOs may not be adequately prepared to engage successfully. However, as the CSO data elucidates regional and local outreach is far more effective in their eyes than engagement held outside of their geographic constituency.

The foundation for engagement: a global perspective

This section combines responses for all IGOs to give a snapshot of global engagement across all IGOs assessed by all respondent civil society actors. The analysis is organised according to the Scorecard’s four dimensions of engagement.

 

Access

This box combines responses for all IGOs to give an overall picture of civil society satisfaction with the broad international arena.[ii]

Civil society views on IGO access

• 80% of respondents felt that CSO access to IGOs was very poor, poor or okay.
• 59% of respondents felt neutral or didn’t feel that CSO access to IGOs had improved over the past five to ten years.
• 71% of respondents felt neutral or that IGOs weren’t proactive in providing ways for CSOs to attend or participate in meetings.
• 71% of respondents felt that there were benefits for CSOs to being accredited by IGOs.
• 51% of respondents felt that the CSO accreditation process was clear and easily understandable.

The percentages offer a snapshot of civil society engagement by IGOs in 2014. There are some interesting findings, with over half the civil society organisations saying that the accreditation processes were clear and easily understandable. This finding is contrary to anecdotal evidence of an overly arduous process to gain accreditation to an IGO.[iii] From the data, it is possible to identify some real needs: (i) accessibility is generally seen as poor, despite efforts by IGOs; (ii) CSOs want more avenues to access IGO decision-making; and (iii) over half of CSOs see the accreditation process as clear and easily understandable. Additionally, 71 percent also see benefits to accreditation illustrating the potential use of accreditation and access to IGOs as a political tool.

In the graph below we see answers to four questions in the access indicator by the geographic focus of CSOs.[iv]

The graphs above plot the responses to four questions in the survey. Interestingly, we see that the median for the question, “Is the IGO proactive in helping CSOs physically attend?” remaining relatively stable across the three geographic regions assessed. However, we also see that CSOs from Africa feel that IGOs are more selective in their outreach in comparison to their counterparts in Asia and Europe. When assessing the graph “IGO is selective in its outreach” no African respondents felt that IGO outreach was either (a) broad or (b) very broad.[v] Taken globally the median for this question clearly demonstrates that across the world, CSOs feel IGO outreach is selective in its approach.

When we asked CSOs “Has access improved over the past five to ten years?” European CSOs felt that IGO access had improved the least.[vi] This may potentially reflect a frustrated approach held by CSOs based in Europe who regularly engage with IGO decision-making hubs. Although, it is important to point out that CSOs based in Africa and Asia felt slightly more positive, potentially demonstrating a greater emphasis on engaging actors outside of Europe over the past five to ten years.

Policy dialogue

The picture of engagement of CSOs in policy dialogue is poor. The overall picture suggests CSOs are frustrated in their desire to engage with IGOs in this way. This feeds into wider critiques of global governance systems, within which consultations are seen as the bastion of civil society engagement. Yet, as previously discussed CSO outcomes from consultations are rarely visible and can lead to an apathetic approach to policy dialogue by civil society.

Civil society views on policy dialogue

• 63% of CSO respondents felt they had very poor or poor impact on IGO policy.
• 38% of CSO respondents agreed or strongly agreed that IGOs listen to them on policy.
• 45% of CSO respondents agreed or strongly agreed that IGOs don’t even pretend to listen to them when it comes to developing their policies.

We can note from our findings that CSOs generally feel negatively about policy dialogue. Nearly two-thirds of CSO respondents felt that their impact on IGOs was negligible; this finding is compounded by the fact that nearly half of CSOs felt IGOs do not listen to them on policy issues. In essence, IGOs are still struggling to engage civil society on the issues which they are mandated to counter.

The results outlined in relation to the policy dialogue indicator demonstrate an issue at the heart of global governance. IGOs are organisations primarily led by the will of member states. Policy dialogue remains one of the most opaque processes in the theatre of global governance, within which, the will to include civil society remains tenuous at best. Despite the advances in civil society inputs into policy dialogue, the Scorecard has uncovered a key area where civil society engagement remains under developed.

The graph below shows geographic variation in average responses to the question, “How much impact has your organisation had in shaping and amending the IGO’s policy?”[vii]

Impact on IGO policy, by CSO region

We see that Africa-focused CSOs gave IGOs much lower scores for impact on policy than the other three. This likely reflects the ‘Northern’ advantage of CSOs based in Europe or the US. This graph reinforces issues of cultural and geographical bias for CSOs based in the global North. Globally, we see the median for the sample around the middle of the y-axis demonstrating that CSOs on average feel they have a neutral influence on shaping and amending IGO policy.

Programme partnership

A common complaint of civil society is that IGOs view them purely as deliverers of projects, so we asked respondents if IGOs engage them at the programme development stage, and if they felt that IGOs are only interested in them for implementation. The table below presents the percentage of respondents answering negatively to positively.  Fourteen percent of respondents, for example, provided strongly negative responses on engagement in programme development.

Civil society views on programme partnership

• 36% of CSOs agreed or strongly agreed that IGOs engaged them at the programme development stage.
• 26% of CSOs agreed or strongly agreed that IGOs were only interested in them for their ability to implement projects.

In the graph below we see answers to the programme development question by the geographic focus of CSOs. Perhaps surprisingly, respondents giving their geographic focus as Europe appear the least satisfied with IGOs on this question.[viii]

IGO engagement when developing programmes, by CSO region

In the graph above, we see CSOs from Europe giving the lowest rating for IGOs engaging them when developing new programmes. Respondents from both Africa and Asia on average provided higher ratings. This is likely to reflect the efforts made by IGOs to engage local actors (in Africa and Asia) when developing new programmes, rather than European-focused CSOs. Globally, with the combined total of all CSOs we see the median for the sample felt neutral when asked if they had an impact on IGO programme development. This finding reinforces notions that IGO decision-making when developing new programmes, on average, falls short of their commitments to civil society engagement.

Empowerment

IGOs often talk about how they work to actively strengthen civil society in their engagement, so we asked two questions. First, we asked if respondents agreed with the statement, “The IGO collaborates on initiatives that matter to us,” and “How empowered do you feel by the IGO?” The table below shows the percentage of respondents giving negative to positive answers. Although the responses mask the differences between IGOs, the overall picture is not as positive as IGOs might claim, nor is it as negative as some civil society actors claim.

Civil society views on empowerment

• 68% of CSO respondents felt neutral or disempowered by their chosen IGO.
• 44% of CSO respondents felt that IGOs collaborated with them on issues which matter to them.

As the results demonstrate, 44 percent of CSOs that participated in the survey felt that the IGO collaborated with them on “issues which matter them.” This can be reinforced by a common theme that emerges when engaging with focal points at IGOs. CSOs and IGOs often have the same objectives and aspirations for macro-level issues. The key point for disagreement is the scale, speed and methods of achieving these overarching objectives.[ix]

However, when asked directly whether they felt empowered by the chosen IGO, 68 percent of CSOs felt neutral or disempowered. Objectively, the basis for any question on “empowerment” is difficult to quantify scientifically; however, the Scorecard is a perceptions survey and thus aims to evaluate how civil society organisations feel when engaging with IGOs.

Geographic disparity

The graph below uses responses to the question on collaboration and categorises respondents by their geographic focus.[x]

Again, we see respondents with Europe as their geographic focus give IGOs the lowest score for collaboration. Interestingly, the graph below demonstrates CSOs based in Africa offer the highest score for collaboration on initiatives that matter to them, and Asian CSOs score higher than European CSOs. This illustrates a level of proactivity by IGOs when engaging with civil society. Moreover, as discussed in the case studies, many IGOs have initiated dialogues and consultation strategies at national or regional level.[xi] Whilst the Scorecard aims to assess access to main decision-making hubs, this evidence could be used to strengthen the importance of localised dialogue strategies and prioritise the filtration of localised dialogue outcomes to the global policymaking level.

Reinforcing diversity in civil society

We also asked respondents about their own interests and activities. The graph below shows answers to the question, “How much of a priority is IGO engagement within your organisation?” Answers are split between CSOs that reported having dedicated IGO outreach staff and those that did not.[xii] Two things are notable. First, for both types of CSO respondents, IGO engagement is important. This suggests the Scorecard may have successfully targeted actors whose activities are relevant to the Scorecard itself. Second, the fact that CSOs without dedicated outreach staff still consider IGO engagement a priority is likely a challenge for IGOs trying to reach out to under-resourced CSOs.

The answers given within this section highlight a trend throughout the Scorecard. Whilst CSOs may place a priority on engaging with IGOs, a fundamental stumbling block to effective engagement lies in CSO resources. A clear shift in priority has taken place, primarily by placing the onus on CSOs to be proactive in their outreach, rather than IGOs. Whilst it is fair to expect balanced attitudes toward engagement, the general resources at the disposal of IGOs far outweigh the capacity of most CSOs.[xiii] Therefore, a key finding from this body of research should be to place a greater priority on IGOs to proactively engage with CSOs, rather than allowing a self-selecting sample[xiv] of CSOs to frequently engage with them.

The graph below shows the geographic focus of respondent CSOs and shows the split between CSOs with (orange) and without (green) dedicated IGO outreach staff. The number for each bar is the total number of completed surveys.

Geographic focus of respondents, by IGO outreach staff

Obstacles

We asked respondents to tell us about the obstacles they face when engaging IGOs. It is notable that accreditation processes were not the main obstacle for respondents. Instead, the common frustrations appeared to be with member states, IGO consultations without outcomes and most notably the inaccessibility of IGO outreach mechanisms. The findings displayed below illustrate the frustrations felt by CSOs when engaging with global governance systems. Interestingly, under resourcing of IGO focal points was the lowest obstacle. This finding also speaks to a potential lack of awareness within CSOs of IGO focal points and their relative underfunding and understaffing in comparison to the size of the IGO and number of CSOs who engage with the organisation.

CSO opinions on obstacles to engaging IGOs

The key obstacle to IGO engagement from a CSO perspective is the accessibility of IGO outreach mechanisms themselves. This finding leads us to the conclusion that awareness of mechanisms remains obscure in the eyes of CSOs. Despite efforts by IGOs to enhance ease of accreditation, we are still seeing a common theme of accessibility within outreach strategies being insufficient to capture a diverse group of CSOs. Moreover, this can be compared directly to the results discussed earlier about selectivity of engagement by IGOs.

 

The IGO perspective

CIVICUS also distributed a self-assessment to the 10 IGOs participating in the evaluation. The questions were based around the same four indicators and were designed as an opportunity for the IGOs to offer an honest reflection on their civil society outreach. Whilst this is only a small sample of all IGO staff that frequently engage with civil society, it offers a glimpse into the perspective of the IGO.

We asked IGO respondents, “What do you think are the main obstacles to greater CSO engagement on your organisation’s policies?”

• Eight respondents said the main obstacle is member states.
• Four said the main obstacle is the IGO’s own leadership or management.
• Nine said the problem is the CSOs themselves.

IGOs were asked to elaborate on the key issues which prevent CSOs from having effective impact within their systems. Below is a sample of the answers that were given.

 

Some comments from IGO staff:

“Little coordination: speaking with too many competing voices…”

“Not a balanced representation – same CSOs leading the process…”

“CSOs do not know our procedures.”

"When working with local CSOs in Africa, capacity constraints quickly become apparent, especially limit understanding of complex technical matters.  Some of them also lack sincerity as there are 'fake' CSOs that try to obtain funding from donors but then turn silent when they should represent citizens' interests in front of governments."

“There is a tension between CSOs as true collaborative partners and the competition between NGOs as 'business' partners.”

“Would rather focus on partnering with operational NGOs, but sometimes the advocacy CSOs get in the way. Organisation is forced to devote too much attention to advocacy NGOs who repeat the same arguments year after year.”

“We don't always hear each other. It would be more constructive if both sides arrived ready to listen, not just ready to talk. Also, some CSOs try to control access to our organisation which seems counter to the concept of civil society.”

"International CSOs cannot always claim to have the best interest of a country's citizens at heart. Several of them highjack meaningful development projects at [the] national level to push advocacy agendas at [the] international level. This can slow down or even terminate preparations of projects that are much needed in countries to reduce poverty and increased shared growth."

Interestingly, member states were a common frustration amongst both IGOs and CSOs. CSOs placed importance on mechanisms and accessibility, whereas IGOs staff ranked management or leadership of the organisations as an obstacle to CSO engagement. Yet, it would seem these two issues are entwined as our research indicates that some IGO leaders or management are far more willing to uphold their commitments to civil society engagement than others.[xv] Shifts in priority are key to maintain civil society engagement as central part of the agenda of IGOs and reinforce the need for structural shifts within global governance. Other IGO focal points spoke of expanding their work not via increased resources, but through dedicated policies focusing on raising awareness in their organisation of best practice regarding civil society engagement.

 

---

 

[i] Please note not all IGOs completed the self-assessment.

[ii] Not all of the questions were answered on a scale of ‘very poor’ to ‘very good,’ but those labels are used here for simplicity.

[iii] However, as explored within the case studies, the accreditation process for assessed IGOs is vastly different.

[iv] The red dot shows the median for that group. The highest and lowest parts of each plot are the minimum and maximum responses. The top of each box is the upper quartile: 25 percent of the data are above this, and the bottom of each box is the lower quartile: 25 percent of the data are below this. Only regions with significant numbers of responses are included. The y-axis represents the five possible responses to this question: 1. Strongly disagree; 2. Disagree; 3. Neutral; 4. Agree; 5. Strongly agree.

[v] When assessing graph “IGO is selective in its engagement” the interquartile spread of the African respondents is significantly lower down the y-axis than the other two regions assessed. The y-axis represents the five possible responses to this question: 1. Outreach is very selective; 2. Outreach is somewhat selective; 3. Neutral; 4. Outreach is somewhat broad; 5. Outreach is very broad.

[vi] The y-axis represents the five possible responses to this question: 1. Accessibility hasn’t improved at all; 2. Accessibility hasn’t improved much; 3. Neutral; 4. Accessibility has slightly improved; 5. Accessibility has significantly improved.

[vii] The red dot shows the median for CSOs of that geographic focus. The highest and lowest parts of each plot are the minimum and maximum responses. The top of each box is the upper quartile: 25 percent of the data are above this, and the bottom of each box is the lower quartile: 25 percent of the data are below this. Responses to the survey: 1. No effect; 2. Minor effect; 3. Neutral; 4. Moderate effect; 5. Major effect. Regions with only a few responses are excluded from the graph.

[viii] The red dot shows the median for that group. The highest and lowest parts of each plot are the minimum and maximum responses. The top of each box is the upper quartile: 25 percent of the data are above this, and the bottom of each box is the lower quartile: 25 percent of the data are below this. The y-axis represents the five possible responses to the question: 1. Strongly disagree; 2. Disagree; 3. Neutral; 4. Agree; 5. Strongly agree.

[ix] This observation is grounded in semi-structured interviews with many of the IGOs assessed in this survey.

[x] The red dot shows the median for that group. The highest and lowest parts of each plot are the minimum and maximum responses. The top of each box is the upper quartile: 25 percent of the data are above this, and the bottom of each box is the lower quartile: 25 percent of the data are below this. The y-axis represents the five possible answers to this question: 1. Strongly disagree; 2. Disagree; 3. Neutral; 4. Agree; 5. Strongly agree.

[xi] Please see individual case studies for further information.

[xii] The responses are shown by percentage.

[xiii] IGO resources refer to resources in the larger sense. It is not intended to refer to resources for civil society engagement teams within IGOs. Civil society outreach department funding within IGOs is relatively small in comparison to other departments.

[xiv] CSOs which have the funding, resources and cultural capital to enlist dedicated staff to be permanently or semi-permanently based around IGO decision-making hubs.

[xv] World Bank staff attribute a key shift in the civil society strategy of the World Bank is attributed to the appointment of Jim Yong Kim. Under his leadership a greater emphasis has been placed upon civil society engagement. This can be further explored. P Stephens, World Bank Moves to Increase Civil Society Engagement, Devex, 14 October 2013, available at: https://www.devex.com/news/world-bank-moves-to-increase-civil-society-engagement-82077 .