Case Study Owner Feedback

Each of the Digital Dialogues case studies was evaluated in its own right, having been run as a distinct exercise. Each case-study had an ‘owner’, who was the principal point of contact between the researchers and a particular case-study. Case study owners tended to be policy officials, though some were from communications divisions. They coordinated a ‘team’ made up of policy officials, communications professionals and, in a few cases, IT delivery staff (who were either directly involved with the activity or providing some routine support).

As part of the evaluation, case-study owners were interviewed and feedback was solicited via a structured survey distributed to those involved in setting up and running the case-study website. General themes materialised from the feedback; these are summarised below. Where case-study specific feedback was obtained, we include it in the relevant report.

CASE STUDY OWNERS ON…INNOVATION

Case study owners tended to be highly motivated by the opportunity to enhance public engagement in their area of policy. Most either discovered Digital Dialogues through their own research or were encouraged by senior managers to get involved in the pilot.

Prior to participating in the project, case-study owners recognised that ICT offered a way of facilitating public engagement. However, a lack of specialist knowledge, concerns about the unclear risks associated with online discussions and the limited availability of resources had prevented them from exploring the extent of this potential in relation to their agency, department or office.

Digital Dialogues provided a space in which case-study owners could explore the potential of ICT in a ‘live’ pilot without having to undertake procurement or without needing to allocate or divert significant resources. Moreover, their work would be evaluated, which could inform future approaches.

...ENGAGEMENT ONLINE

Case study owners understood different things by the term ‘public engagement online’. For some, it was primarily about communicating key policy messages to the public; the use of ‘next generation’ web technology came into play in that it allowed the public a ‘right to reply’. Others wished to experiment with tools that would provide an additional route for public responses to government consultations.

For others, the proliferation of ‘read/write’ web technologies coincided with an aspiration to engage the public in a more constructive and ongoing manner than had previously been achievable through the use of established, mainly offline, consultation methods. Applications such as blogs, forums and webchats, they suggested, were a means for government to put its case forward, for the public to put across theirs and for there then to be a dialogue on that basis.

This latter approach to encouraging the participation of the public in policy making was important to these case study owners as a way of raising confidence in government, promoting active citizenship, and enhancing the quality of policy making by engaging with a broader participant base in a deliberative environment. These case study owners believed this form of engagement would be a departure from the norm:

The public are distrustful of formal consultation exercises so it doesn’t help public confidence. Both the public and government officials need to be re-educated about the role the public can play in policy making… Technology can help government think of more creative ways of involving the public in the different stages of the policy making process. It is quicker, so consultation can be more responsive and potentially more public participation exercises can take place. I would hope that as a result the quality of policy making improves and the public feel genuinely involved in the policy making process and understand how they can contribute.

...ONLINE CHALLENGES

Case study owners were aware of some of the challenges they faced when using online tools. Prior to launching their pilot websites, they expressed concern about the reception they would get from the public: would it be hostile? Would anyone want to talk to them? They accepted their inexperience of using the web as an engagement tool and that government had been – to date – a ‘silent partner’ in the development of the UK’s online polity:

I think that potentially it could help to create a much greater feeling of empowerment on the part of citizens. People have much greater access to information, and are becoming increasingly sophisticated in their relationships with sources of information, whether this is the mainstream media, the government, or the new communities that are being created online. Whether they agree or disagree with government policy, the public are now in a much better position to engage critically with it. I believe that this is generally a good thing for democracy.

There was concern that motivating and maintaining the interest of the public would be difficult because there would be a level of cynicism about government’s commitment to engaging online, and because government would be competing for people’s attention in an already busy world wide web.

Cynicism they could deal with, but what made the case-study owners most apprehensive was that their first attempts to use the web for engagement would be hijacked by interest groups or vandalised with abuse. Given the nature of their policy areas, some of the case-study teams were also concerned that the policy or discussions might be too technical for non-specialists and this complexity might be viewed by some as a deliberate attempt to discourage participation.

...PARTICIPANT NUMBERS

All the case study owners reflected that although they had not known how many participants to expect, they had envisaged more contributions than were eventually received. Most, however, welcomed the low numbers given the limited resource availability for these exercises and their own levels of experience.

Although they had envisaged more participants and more deliberation, in their feedback the teams also said they had underestimated the time required to moderate and facilitate the sites. It took time to read through the submissions, and although they did not respond to every single post, making sure that their responses were of sufficient quality also meant that officials had to take time to research and compose their own contributions. Some case study teams also reported that they saw a direct link between their participation and levels of participant interaction, return visits and the quality of contributions.

...USER DEMOGRAPHICS

All case-study owners reported that group and individual stakeholders with whom they had already established contact did not take part in the online deliberations. In some cases, the case-study teams invited subject experts and specialist stakeholders to participate who, it was hoped, would deliberate alongside the lay public. A few did log on and made positive contributions to the exercises, providing participants with the benefit of their knowledge and raising the bar in terms of the quality of submissions. Most, however, did not take up the invitation, to the disappointment of the case study owners. Reflecting on the reasons for this, the case study owners believed that the experts and stakeholders did not perceive value in these new routes over that which they derived from established means, such as calling, writing a letter, submitting a formal consultation response paper or even simply using email.

...USER GENERATED CONTENT

Many participants made well-argued and carefully researched contributions to the online deliberations, which policy officials found insightful and useful. However, the quality of submissions was not consistent and case study owners complained that they often read posts that were irrelevant, badly structured or which made generalisations based on personal experience: the ‘media effect’ was also cited as an influence on public opinion, with submissions often seeming to reflect dominant headlines.

It was not felt that the majority of users were abusing the opportunities presented by the case study websites. Case study owners believed that those taking part felt connected to the issues being discussed but did not necessarily know how to get their views across effectively. This, in turn, meant that the policy teams found it difficult to respond or to find ways of incorporating such submissions into their own policy making processes.

Some of the case-study owners felt that just as they received guidance on how to contribute to blogs, forums and webchats, there was a need to provide citizen participants with similar tips on submitting responses. Encouraging more use of background facts and figures was highlighted as a way to deal with this problem (though not all sites made such information available for participants).

...INFLUENCING POLICY

Despite there being some factors which disappointed the case study teams about the interaction with the participants, it was recognised that the online routes being piloted were bringing people and communities into the policy process who had not previously participated. These new participants had the potential to bring new experiences and information into the orbit of the policy officials, and this was identified as the most conspicuous advantage of online engagement.

As part of the evaluation, case-study teams were invited to talk about how they used the input they received via the sites. Each said that it was too soon after the activity to say for definite what the impact had been on policy or their engagement processes. In terms of policy making, case study owners cautioned that although there were benefits to be found in the immediacy of online engagement, it did not necessarily mean that the result would be faster policy making. But they said it did, perhaps, enhance openness and transparency and therefore, the quality of the policy making process.

Within the duration of Digital Dialogues, there were some cases where blog, forum and webchat contributions were added into consultation response documents to provide insights into public views behind the statistics. In other cases, where there was not a paper or report, contributions were collated and distributed to relevant policy teams to provide them with experiences, ideas and views they might not otherwise have had access to.

It follows that, in some instances, there was a direct and short-term application for the material derived from the online channels and, in others, that material was ‘banked’ with the potential to have a more indirect, long-term influence on policy making. In either circumstance, the case-study owners emphasised that the online input was treated as credible data and not viewed as being any more or less valuable than that which had been received through other channels.

...PROCESS MANAGEMENT

In terms of process, some interesting common themes appeared relating to what worked well. It was made clear that while engagement needed to be based on good communications principles, the activity had to be led by policy officials with the depth of subject knowledge. The dynamics of online engagement were felt to benefit from a ‘short chain of command’ that allowed the managers of the activity to deliberate in a responsive and substantial manner:

We feel that a short chain of command helped us get the most out of the forum. We had senior commitment to the general principle, but day-to-day control was delegated to us. This helped us evolve the content over time, post things as they were necessary and change the topics if we felt something more appropriate was needed. We still made sure more than one person looked at everything before it went up, but this was about getting a second opinion, not about ‘sign off’. We found it was important to keep the momentum going on the forum and for participants to know we were listening.

Therefore, to ensure these conditions were present and the resources were in place, successful online engagement exercises were those that established close and efficient coordination between communications, delivery and policy teams. Indeed, the Digital Dialogues research team were struck that, through these small practical pilots and the emphasis placed on good collaborative processes, the case study teams made new connections with their departmental peers and structures:

I think the team worked really well together, especially as it was all done at short notice (about 2 weeks), so no-one had had the chance to allocate time to this work, but it all still got done.

The themes above were common across all the case studies, but feedback also revealed insights relating to the dynamics of particular applications.

BLOGS
Those running blog-based pilots saw some distinct advantages in the medium. Blogs were easy to manage in a technical sense and they were popular (and therefore familiar) amongst the public. Blogging encouraged brief interaction but on a more regular basis and in a more conversational tone. This was appealing to case-study owners with limited resources who wanted to talk directly with the public about developments and issues that might not have been carried by the media.

The risk, however, lay in the ‘culture’ of politically-orientated blogging. The case-study owners perceived this to be confrontational and anti-establishment: they were concerned that their blogs would not be welcomed on account of their government authorship.

FORUMS
Case study owners who used forums highlighted that their asynchronicity was key to their decision to use this particular application over others. Large groups of participants could be brought together without needing to be in a particular place at a particular time. The format of forums was also seen to be an advantage, in that they allowed teams to structure questions and themes in a comparable way with conventional methods, but also enabled them to encourage an open dialogue which was not possible with conventional tools.

Deliberation was difficult to manage, the teams said. They warned that the time and effort required to moderate and facilitate the forum community should not be underestimated. They also found it difficult to know when to step into deliberation. Case study owners said that they tried to get a feel for the rhythm of discussion and to know when not to intervene so as to allow space for peer-to-peer discussion between participants:

We interacted with participants by posting an initial comment from the team at the commencement of each new theme. When there were some participant comments, we would post a new comment, which may be either an ‘administrative moderator’-type comment or a more ‘facilitative moderator’-type comment. The former attempted to keep participants on the relevant topic, keep the tone respectful and respond to any direct process queries. For example, in the opening topic I responded to a query one participant posted about why the forum is moderated. The latter attempted to summarise the views expressed, ask supplementary questions to the whole forum to further explore the issues or respond directly to particular posts. We also posted additional factual information as separate posts when we felt this was necessary for an informed debate.

As moderators, the case-study owners also felt that their role was not adequately explained and, at times, participants had different interpretations. For example, although the moderators were in a position to explain current policy and practice, they were not in a position to speculate on policy change following consultation.

WEBCHATS
For the teams using webchats, there was a concern that the numbers of people involved and the depth of deliberation – given the ‘real time’ dynamic – made webchats of limited value to the policy process. They were, however, identified as having communications value. They could be run as one-off events, to pique interest and help deliver key messages about departments, policies or ministers.

SUMMARY

Whether they used a blog, forum or webchat, all the case-study teams were enthusiastic about using online methods in the future to bring the public into the policy making process. However, they all said that more pilot work would need to be undertaken to build up experience and refine processes. It was acknowledged by all that technology should not just be used for technology’s sake, and that it should be deployed strategically as one part of a multi-channel approach.