End-To-End Processes

Posted by Barry Griffiths on September 28th 2007

The following material provides an overview of the key steps of an online engagement exercise. This has been informed by drawing on existing practice at parliamentary and local government levels, and the experiences of the central government case study owners in Digital Dialogues.

The steps have been divided into three sub-sections:

  • Pre-launch;
  • Live;
  • Closing.

Under each step, key considerations are outlined. These are deliberately designed to be broad and generic in order to cover the range of techniques, tools and users potentially involved in an engagement exercise.

PRE-LAUNCH

1. Aim and Objectives
Before selecting technology or committing to a public or stakeholder engagement exercise online, first define the aim and objectives.

  • What is the overall point of the exercise?
  • What manner of participation are you offering the public and for what end?
  • What are the specific outputs and outcomes you wish to create from undertaking the exercise?
  • How does this activity complement offline strategies?

It is possible to become distracted by the flexibility and reach of online engagement. Getting aims and objectives as clear as possible at this stage will benefit all subsequent decisions.

2. Planning
This stage involves setting out the format of the exercise. The more planning and lead-in time there is the better.

  • Who do we want to consult with; when; and on what basis?
  • Which type of application or combination of applications could be used?
  • How will the online activity be drawn into the consultation or policy development process?
  • What influence are you prepared to give to participants over the eventual outcomes?

Bring all required project staff and partners together at this point and hold a meeting to scope out this and subsequent stages. Encourage colleagues to discuss benefits and risks, and to bring in previous experience.

3. Performance indicators
Consider at early stages what performance indicators you will use. These should be based on the indicators used throughout the broader exercise.

Remember that many website statistics can be misleading. Disregard ‘hits’ and concentrate on registrations, posts/comments, unique visitors and repeat visits. Page rankings in search engines are also useful indicators of the penetration and visibility of your consultation online, so carefully determine what search words the public and stakeholders will use to find the site.

Monitor the number of incoming/outgoing links. It is important to remember that those sites that are most visible online and highest in search engine rankings are those that link out to other sites and receive reciprocal links. Particular efforts should be made to network with other relevant online communities and resources.

Ultimately, web statistics make for inadequate key performance indicators but provide useful contextual data. Much more substantial are indicators relating to who participated and what happened as a result.

4. Build management
This relates to which application or medium is being used. To recap, in Digital Dialogues we found:

  • Blogs are useful for ongoing, low-intensity consultation;
  • Forums are good for episodic usage;
  • Webchats are useful for one-off event-style stakeholder engagement.

Key considerations prior to committing to the construction of a website are:

  • Can you build an application or site in-house, or do you need to hire a contractor?
  • Do you need to build/buy in something new or reconfigure an existing tool?
  • What are your functionality requirements?
  • Do you want to use open-source or proprietary software?
  • Can you manage the build within your team, or is there a need to draw in expertise from other teams?
  • How long will it take?

Hosting and database requirements should also be considered here. You will find it beneficial to draw on internal expertise from your IT/web team.

5. Design
This is an important aspect in ensuring that your platform is conducive to deliberation, appeals to participants and draws them in on more than a fleeting basis.

Design will be largely dictated by answers to the questions asked at planning and project management stages. However, it should also draw in considerations about inclusivity, accessibility standards, existing branding and sustainability (in the sense of how quickly the design will date). Above all, ensure that design is user-centred.

Consult existing branding guidelines to ensure you are meeting departmental requirements.

6. Copy/Content production
Choices about vocabulary and syntax are crucial to a successful consultation online.

  • What are the key messages and priority questions?
  • Do these need to be adapted from a consultation/policy document for publication on the web?
  • At what stages of the consultation could/should content be updated?
  • Whose responsibility will this be?
  • Is there a lot of technical language? Does the site need a glossary?

Remember that copy online works differently to offline. More often than not, it should be shorter and more direct. Testing copy on internal or external focus groups ahead of launch is a sensible investment.

7. Resources
More often than not, background information resources will be required by the participants to enable and inform participation.

  • Are these provided on the site or hosted on other sites and linked to?
  • Are they sufficiently visible on the site and are participants encouraged to make use of them?
  • Will there be a dedicated page for holding resources or will they be downloadable from specific participation points?
  • Have all relevant documents been made available for download?

8. Legal
Always ensure that consultation rules and guidelines are available to participants.

  • Does the site meet the required standards in the Cabinet Office Code of Consultation?
  • If relevant, does the site meet the required standards in the Ministerial Code?
  • Does the site meet the required standards in the Civil Service Code?
  • Does data capture meet Data Protection and Freedom of Information requirements?
  • Are accessibility standards observed?

If in doubt about any of these questions or copies of codes, contact the department’s consultation coordinators, legal team and/or web team.

9. Technical support
Technical problems may arise for the administrators of a site, which are often quickly spotted, or for the users, which often take longer to be drawn to the attention of those who can rectify them.

  • Is your support being provided by a departmental team or an external contractor?

Technical risks should be reduced by careful procurement and pre-launch testing. During the live exercise, technical support should also be on-call to deal with issues within at least a 24-hour period.

Provide an email address that users can use to report problems.

Keep a record of technical problems experienced over the course of the exercise.

10. Testing
Testing can seem like an unnecessary hassle that can be left as a responsibility for a supporting technical team or dispensed with where time is in short supply. This would be misguided; testing should always be carried out, and it should be task-orientated.

Wherever possible, testing through simulated exercises should involve those communications, policy and web teams who are scheduled to be involved when a site goes ‘live’.

These ‘dry-runs’ are vital if everyone in a combined team is to work adequately together once the site is active and being utilised by members of the public. Testing will also highlight technical bugs that only present themselves under ‘live’ conditions – potentially saving embarrassment at a later date.

11. Marketing
Communications teams should be consulted on this aspect.

Promotional activity will be driven by the focus of the exercise and the desired user base; it should also reflect established procedure in the department and across government. It should be planned within budget, time and be designed to meet the objectives of the exercise.

A balance of direct marketing and media relations techniques works well. Give consideration to how the marketing and publicity of the consultation will be managed as it progresses and once it has closed.

Remember, it will not be enough to do one push at the launch of the consultation; marketing can be staggered but it must be ongoing.

Research your target participants extensively. Find out socio-demographic data about them, their attitudes and behaviours. Make a particular effort to access information that covers their use of the internet and political engagement.

12. Recruiting users
Marketing is vital to recruitment and is tied in with the early defining steps. It is important to consider recruitment throughout because in some instances recruitment will be an ongoing process.

You should also consider:

  • What personal information do you want to gather? Will this meet data protection standards?
  • Consider what types of people you want to recruit – is there a specific demographic or do you want to get a mixture of demographics together?
  • Do you want to broaden and deepen your stakeholder base, or focus in on expert practitioners?
  • If you want to bring expert stakeholders and the public together in deliberation; will this be their first meeting?
  • Are your participants in a ‘hard-to-reach’ group?

LIVE

13. User community management
Key considerations concerning management of a community of participants are:

  • Are participants registering properly?
  • Are they providing the necessary details?
  • Are these details being stored?
  • Are complaints/problems/positive feedback being dealt with?
  • Are you updating participants regularly on any significant milestones over the course of the consultation?

14. Moderation
‘Moderation’ refers, here, to publishing participant contributions. It is a crucial but flexible aspect of all online engagement.

Your approach to moderation will depend on which platform you are using. Weblogs require the least moderation, forums the most, although again this depends on other factors – for example:

  • Will you be moderating posts before or after they go live?
  • Who is using the site – have they been consulted before and has this taken place online?

All of the Digital Dialogues case studies employed a pre-moderation strategy (comments were checked against the site rules and for relevance before publishing). On some occasions the Hansard Society moderated comments but it is preferable for the owners of the exercise to moderate. Moderators were required to check for new comments at regular intervals (at least four times a day).

Transparency in moderation is a very important component of successful engagement online. A moderation policy – outlining the what, why, when, who and how – should always be provided for participants to read.

15. Facilitation
Like any offline meetings or stakeholder engagement, online participation also requires good chairing. This is the single most important aspect of online consultation that is deliberative in nature. At least two members of staff, preferably policy officials, were assigned to an exercise during Digital Dialogues.

Participants will deliberate amongst themselves, but the participation of the government representatives is the glue, and in its absence the participants will lose interest and become frustrated. At best, participants will voice their criticism to you; more likely they will drift away and tell others about the experience.

  • Which members of staff or departmental representatives will post in the flow of the deliberation, keeping the focus, asking further questions, responding to queries?
  • How many people will be assigned to facilitate the discussions?

16. Summarising
Considerations of how and when to summarise deliberation are most relevant to forums, where asynchronous group-based deliberation is taking place. Summarising is as important for ‘veterans’ of the deliberation as it is for ‘newcomers’. It is also recommended for the benefit of the moderation team and content analysis at the close.

As a guide, the more regularly the policy team visit the deliberation (not necessarily always to participate) the more efficient and constructive the post-activity analysis will be.

It is also good practice to offer participants an opportunity to review the summaries and make queries or suggestions for inclusion.

CLOSING

17. Archiving
Your platform should automatically archive the user-generated content, participation data, and all accompanying analytics. However, it is important to consider how this automated archive will be taken offline, stored and accessed by your team.

  • Which data elements will be shared with the public and at what points?

If the intention is to use the site again for a follow-up exercise:

  • Who will the content be changed by and how?
  • What will the site be used for in the interim?
  • If the plan is to reuse the platform for another consultation, how should the content and databases be cleared and should the platform be shifted to another server or the URL redirected?

18. Analysis and Reporting
Online participation exercises gather a great deal of data – the submissions, the participant details, site and server analytics. This aggregation and ability to filter this data set is one of the foremost attractions of online engagement tools.

The considerations for this stage relate to:

  • Who will be responsible for analysing the data, at which stages and when is the report deadline?
  • What are the key indicators and how will these be related back to findings from other methods employed for the exercise?

At the end of each exercise it is good practice to provide – as a package – transcripts, an executive summary and a statistical report.

You should consider who will compile the report and who it will be distributed to (it is best practice to post the same report on the website for public access as it is passed on to those conducting the exercise). The length of the report will depend on the focus, participants and duration.

Consider asking participants to review the report. Provide a deadline and request comments on omissions or clarifications. Retain editorial oversight but do give genuine consideration to suggestions.

19. Response management
It is important as soon as the exercise closes to explain any next steps to the users. You do not need to present conclusions or definitive findings at this stage, but it is important to manage expectations. Consider when you will be able to make a ‘final’ response, who will make it and where it will be distributed from.

Not having sufficient feedback processes will frustrate users and discourage participation in future engagement activity.

20. Evaluation
It is good practice to conduct an evaluation at the end of any public engagement activity; online engagement is no different and during these formative stages is crucial.

The purpose of the evaluation is to look back at the aims and objectives you set for the exercise and ascertain whether or not these were achieved. The evaluation should pinpoint the factors contributing to the success or lack of it. For example:

  • Was planning time sufficient?
  • Was the application fit for purpose?
  • Did the marketing transmit the purpose of the exercise to the target users?
  • Were project costs adequately managed?

It is acceptable that an evaluation can remain internal, but consider the value in also making the evaluation available to the public, or at least the participants. Other agencies, departments and ministerial offices are also likely to benefit from your experiences.

Evaluation of the online engagement activity should be included within impact assessments of the broader engagement exercise.

For more detailed guidance on the process points above, consult the ‘Useful Resources’ section of this report.

up