Cross-Case Study User Feedback

The second phase of Digital Dialogues piloted 12 case-studies – the unique features of each are described in specific case study evaluation reports in the previous section of this report (2.3). However, common themes were identified across the sample and these are discussed here.

Through the Digital Dialogues case studies a number of interesting themes appeared – some of these relate to demographics, others to the experience of using a particular web application for deliberation. As elsewhere in this report, users are defined as those who visited, registered or participated in the website; case-study owners are those who developed the online deliberation.

DEMOGRAPHICS

Across the case studies, users consisted of a mix of demographics. They were a range of ages, ethnicities and genders. Though, in general, men were more likely to use the sites than women; those between the ages of 25 – 60 years old were the most active.

Men were more likely to get involved in online deliberations than women, although certain case-studies appealed to larger numbers of women (e.g. Family Courts and the European Youth Parliament) and others were disproportionately male dominated (e.g., Party Funding Review forum) despite attempts to address gender bias during recruitment.

Although the case-studies attracted users from a range of ethnic backgrounds, the majority defined themselves as British or English:

Most users (96%) regarded themselves as regular users of the internet (n=1559). The majority of users accessed the internet from home:

POLITICAL ENGAGEMENT AND ATTITUDES

In general, users were more politically engaged than the average UK citizen – particularly with regard to the issues under discussion – even before registering with the website. For example, 85% of those taking part in the Party Funding website had voted in the most recent general election; 43.5% of those visiting the Food Standard Agency’s blog had written to their MP.

Despite this, efficacy prior to participation was low amongst most users of the case-study websites. The exceptions were the Planning Portal and the FCO–EYP forum (where one group were government employees and the other were associated with a political-interest organisation). Particular websites attracted more sceptical users; those taking part in the Family Courts forum, for example, were less likely to believe that policy makers would listen to them than those taking part in the ONS blog.

Most users said they had an existing interest in the subject matter being raised in the case study they took part in. This was borne out in the routes people cited that they took to get to the online engagement activity. These were often established networks and sources, such as email newsletters and departmental websites.

Despite being comfortable with technology, interested in the subject matter and even having undertaken some form of political participation previously, for the vast majority of users taking part in a Digital Dialogues case study was their first engagement with a government policy consultation exercise. These users had never engaged with the policy process through a formal route until an online option was made available.

For most, online engagement was their sole means of interacting with the consultation, with only a quarter contributing by another means where it was made available (for example, by attending a citizens’ jury, writing a letter or sending an email). Survey respondents highlighted a number of advantages they saw as being unique to online engagement over other available routes. These included:

• Taking part anonymously;
• Taking part without having to be in a particular place at a particular time;
• Forming an opinion over time;
• Finding out what other people were saying;
• Engaging with government representatives.

USERS ON USER EXPERIENCES

Each case study attracted a high number of unique visitors who made repeat visits. However, most of these people took part only as spectators (this reflects the usage trends of online engagement generally). Despite this, the majority of users believed that online methods are a credible form of engagement:

Of those providing feedback on this measure (n=639), 74% said that they were useful, whether or not many people contributed to discussions. Similarly, 78% said that they would engage with the government online in the future (n=484). A similar number said that they would recommend online engagement to others, and in open-ended questioning, many suggested that the rate of involvement is likely to grow as opportunities become more widespread and regular in their occurrence.

...ON MOTIVATIONS
In terms of their motivation for using the sites, users arrived with mixed goals. They wanted to have their say on the issue or policy, but also to learn about the experiences, knowledge and ideas of others. To engage in a two-way exchange of information was the core dynamic driving these exercises.

Few took up the opportunity to actively contribute, and the majority chose instead to visit the site and read the submissions made by others, rather than make their own contributions. However, as noted earlier, community websites of all kinds receive more spectators than contributors, and spectating is a credible (if passive) form of engagement.

...ON MODERATION
On average, about half of the respondents felt that they had not learnt anything new about the policy area, process or the sponsoring department from the contributions of case-study owners. Users also expressed disappointment with the quality of discussions that took place overall. Most perceived that the majority of interaction was between participants and that there should have been more frequent contributions from policy officials and government experts. Consequently, users said they learnt more from other users than government.

In each case, policy and communications staff were visible on the sites and made contributions. On each site, the nature of their participation was explained as ‘moderation’. Interestingly, this was taken to mean different things by different users and a range of opinions emerged about what function these moderators did or should fulfil. These included:

• Chair or facilitate the debate;
• Check comments for suitability (against the discussion rules);
• Provide knowledge and respond to questions;
• Account for policy implications;
• Represent the views of government.

...ON PEERS
Although our surveys did not expressly ask for the views of users about their peers, feedback of this nature did come through when we asked about advantages and disadvantages of online engagement activity.

Users were asked to state in which capacity they were taking part; those who replied, in the main, said they had a direct stake in the policy area but were engaging as private individuals.

Users perceived others – particularly those who posted – as articulate and well-informed but also opinionated and self-serving. This lead to concerns about the representiveness of the user-base, something users said could be addressed by better publicity (which across all the case studies was regarded as being poor).

Anonymity, while welcomed by users, was seen to have a potentially detrimental effect: it could allow people to misrepresent themselves. Meanwhile, some users were concerned that the government would not take their views seriously because of the informal nature of anonymous online deliberation. Conversely, some users emphasised data-protection issues and the possibility that online deliberations and the associated surveys (whether evaluative or linked in to consultation) were being used to track individuals.

User surveys also revealed insights relating to the dynamics of particular applications:

...ON BLOGS
Only a third of those who used the blogs in our sample were bloggers themselves. However, about three quarters regularly visited blogs with policy or political content. When comparing the government blogs to others they used, the users rated the blogs’ content as below average, and the interactivity of the bloggers as significantly below that of what they were used to on other blogs.

However, overall, blogging by government ministers and officials was welcomed. Users were pleased to have an additional means of accessing information about departments, individuals and policy areas, and a new means of interacting with government.

Thinking long-term, users recognised that government was new to blogging and that there were many ways in which it could develop given the time. However, there was scepticism about the sustainability of government blogging after the initial public relations value had run its course.

...ON FORUMS
Forums are good for engaging large groups on an asynchronous but structured basis. Feedback from the users suggested that the forums in our sample achieved a mixed record of success.

Users were drawn to the forums because they offered an opportunity to directly contribute their own views to the policy process. Nonetheless, the majority were also motivated by the prospect of engaging with experts and other stakeholders and hearing about a broad range of experiences, ideas and opinions.

Users expected a high standard of detailed deliberation. Yet, they also appreciated that, on account of the asynchronous dynamics, the discussions would have an elasticity that would make them difficult to follow. Structure and facilitation were, therefore, identified as being of great importance to the forums. Unfortunately, it seemed that in most cases the interaction from moderators was seen as too infrequent, and the topics and questions were regarded as being inaccessible.

Fundamentally, these users wanted to be listened to and, on the basis of being invited to participate, expected to have some form of influence on the policy process. The extent of this influence and the link between this exercise and the policy decisions was unclear, and this made users sceptical about participation. Almost all said that they expected feedback on what would happen with their contributions.

...ON WEBCHATS
Users recognised that webchats provide a unique opportunity to engage in ‘real time’ interaction with a government or policy representative. Although many saw webchats as an opportunity to express their ideas, few expected to have their questions answered. In the event, almost all the questions posed were responded to: this was a pleasant surprise for the users. Some, however, expressed doubt that the ministers or policy representatives who were supposed to be answering questions were actually doing so: they believed that other people were responding on their behalf.

SUMMARY

What we can see is that the public are keen on the government engaging more online. Yet, it is not clear that the government doing more online will result in greater levels of active participation in the policy process. Instead, from the feedback, this seems to be more dependent not on the quantity of government’s online engagement activity but its quality.

Users are clearly proficient and regular internet users. It is likely that these users are making extensive use of other public and private services online and have become accustomed to certain levels of interaction and quality. Government needs to accept that it is playing catch-up and be mindful that it has to make an extra effort to adhere to established ‘rules of engagement’ online. At the same time, the public should welcome government’s activity over the last year, and while encouraging them to step up their efforts, should be forthcoming with altruistic and clear feedback on what government could do better.