beta.govt.nz — User research (Rounds 1 and 2)

About this report

This is a report completed for the beta.govt.nz site as it was in October 2013. Since this research was completed we've already made some changes to the beta site, so some of the recommendations in this report have already been addressed.

The bulk of this report was prepared by Optimal Experience who conducted the research on our behalf. At various places in the report you will see comment boxes titled "What we think" or "What we've done" and each of the screenshots we've included has a caption. These comments and captions are the views of the project team not Optimal Experience and have been added to the report.

Contents

  1. Executive Summary
  2. Tasks
  3. Participants
  4. Detailed Findings

Executive Summary

Overview

In October 2013, Optimal Experience undertook two rounds of user testing with 11 participants. A beta version of the govt.nz website was tested.

The objectives of this project were to find out:

  1. What search terms do users use in Google and how does landing, following a Google search result, on a deep content page effect the users’ performance?
  2. When and why do users use the local site search, what search terms do they use and what problems do they encounter?
  3. What pathways do users take when approaching a task?
  4. Do users use the ‘Browse.govt.nz’ and ‘Contact government’ (aka ‘Government directory’) sections and do they understand how they work?
  5. Can users easily find information about Departments and Agencies?
  6. Do users have problems with the different information hubs on the site?
  7. Can users find contact details easily?
  8. Do users notice when content links to other agency websites and are they comfortable with that?
  9. What do participants make of the ‘no frills’ design?

Participants

The 11 participants were:

  • Individuals with
    • high (7),
    • medium (3) and
    • low (1) internet usage,
  • 5 Females and 6 Males,
  • Aged
    • < 20 (1),
    • 20-29 years (3),
    • 30-49 years (4),
    • 50-65 years (2),
    • over 65 (1).

User Testing Process

The user tests were carried out at Optimal Experience’s Wellington Office on the 8th and the 15th October 2013.

Participants were given several tasks to complete using the beta.govt.nz website. Most participants completed all nine tasks while some, due to time constraints completed only seven to eight tasks.

Some tasks were slightly altered between the two rounds of testing, but remained essentially the same.

Participants were asked to start some tasks with Google, but otherwise were given the choice between browsing and using the local site search.

As part of the testing session, participants also completed two emotional card sorts. The goal of these exercises was to capture users’ perceptions of NZ government sites in general, and the govt.nz site in particular.

Usability Findings Summary

The user testing revealed

  • 17 positive findings,
  • 1 severe issue that requires immediate attention,
  • 12 major issues requiring urgent attention,
  • 11 minor or cosmetic issues which are less pressing,
  • and 23 observations which are not problematic but important to note as well.

System Usability Score

The website achieved a System Usability Score (SUS) of 68.6%. A score of 68% is considered to be average.

Emotional Card Sorts

Adjectives most participants associated with NZ Government sites (based on the Emotional Card Sort):

  • Helpful,
  • Ordinary,
  • Professional,
  • Usable,
  • Friendly,
  • Organised,
  • Reliable,
  • Trustworthy,
  • Comprehensive,
  • Convenient,
  • Inconsistent

Adjectives most participants associated with the govt.nz site (based on the Emotional Card Sort):

  • Comprehensive,
  • Usable,
  • Straightforward,
  • Helpful,
  • Organised,
  • Trustworthy,
  • Professional,
  • Simplistic,
  • Secure,
  • Convenient,
  • Reliable,
  • Ordinary,
  • Impersonal,
  • Dull,
  • Boring,
  • Friendly

Tasks

All participants who attempted Task 2 (Get out of jury duty), Task 3 (Learn about your consumer rights), Task 5 (Change the name on your driver licence) and Task 6 (Find out when school holidays are) completed those tasks successfully.

Only half or less than half of the participants completed Task 7 (Guidelines around travelling overseas when receiving accommodation supplement and childcare subsidy), Task 8 (Complain about a government agency) and Task 9 (Find out who the Minister is for a specific government agency) successfully.

Key Findings and Recommendations

Most participants liked the site

Most participants said that they would be quite likely to use the site.

“I would use this site frequently — it’s much simpler — it’s like a little government Google. Without having to look at all those horrible busy government websites.”

“Hmmm. Now I know its here would use it more.”

“More likely than I expected, because it covered more stuff than I thought it would.”

Would still use Google

Some said they would still use Google, but would go to the site if it came up as a search result.

“If I went thru Google and couldn’t find it I’d use the site.”

“Google is easier. I think it gives me what I need. If I needed to come here I think it would start with Google.”

“I actually wouldn’t use the website I’d go through Google.”

“I would use Google and wouldn’t bother using a third party.”

Recommendation

Make sure that the purpose and goals of the site are clear.

What we think

We're OK with this observation. We know Google is going to be the preferred starting point for most users, our challenge will be getting our content to be one of the pages people visit after they've done a Google search.

Didn’t notice calls to action

Participants seemed to find it hard to differentiate between content that was just informative and content that required action.

“It was frustrating at times as I could not find links to where I should go. The fishing one and the going overseas, I couldn’t find it.”

Recommendations

Ensure that it is clear what users should do next.

Consider using consistent and specific formatting to highlight calls to action.

What we think

We're going to try a few different ways of providing links to people and testing which ones make them more obvious. We've also changed the styling of links on the site to make them visually stand out more from the surrounding content.

It was not clear when a link was an off-site link and participants seemed to be surprised when they ended up on another site or when they expected to be redirected to another site but didn’t.

“People are not here to enjoy themselves they are here to find information.”

Recommendation

Ensure that it is obvious when a link will direct users to another site.

What we're going to do

We're going to add an 'external link' icon next to the links that take users to another site. For the current version of beta.govt.nz this means a lot of the links will need this as the content we have is thin and relies on users being referred on to other websites. We will test the updated design with users to make sure it doesn't become cluttered.

Hard to find Contact government and Browse govt.nz

Most participants didn’t notice these sections a first, but once discovered, they found them really useful.

Recommendation

Consider providing more prominent access to these sections.

What we think

We've already made some changes to the homepage to make some of these sections a bit easier to find. We also know that a lot of people won't see the homepage at any time during their site visit, so this might not actually be a problem. If users can find the information they need without browsing around the various sections of the site that's a good thing.

Unsure how to navigate the site

Participants relied heavily on keywords and phrases to find their way around and got confused when they couldn’t see an
appropriate keyword or phrase. Some participants struggled to return to the homepage.

Recommendations

Ensure that important keywords are included in headings and section descriptions.

Ensure that users can search by keywords.

Provide multiple paths to information.

What we think

We're going to make some adjustments to the information architecture and we're looking at the feedback we're getting from users when they tell us that something isn't in the right section of the site or it's hard to find. We're also going to do some fine tuning of the site's search engine before we finish this phase of the project.

Design is plain

Most participants thought the design was plain and ordinary, but thought that this helped them achieve their tasks.

Recommendation

Make the site more attractive without adding too much visual noise.

What we think

We've been very deliberate in our approach to the visual design for the site. Keeping it plain allows the content to take the primary focus. We've noticed that people who look at the homepage and see the site as an example of design think it's plain — but when users actually use the site to complete tasks, they appreciate the uncluttered look and feel. We'll continue to test this, but design that supports task completion will always take precedence over aesthetic appeal.

If every things else fails, they would call someone

Most participants said they would call somebody if they had a problem or wanted further information. Some participants
didn’t see the contact information in the right hand panel.

Recommendations

Ensure that contact and phone information is clearly visible.

Consider keeping the right hand panel always visible, even when users scroll the page.

What we think

We've already moved the contact links from the bottom of content pages to the right sidebar. We're going to test if the consistent placement of contact information is enough to help users find the phone numbers and email addresses they're looking for, or if we need to do something extra to the design to help them complete their task.

Tasks

Tasks attempted by users

  • Task 1: Find help for low income and paying your rent
  • Task 2: Get out of jury duty
  • Task 3: Learn about your consumer rights
  • Task 4: Find contact phone number for Ministry of Fisheries
  • Task 5: Change the name on your driver licence
  • Task 6: Find out when school holidays are
  • Task 7: Guidelines around travelling overseas when receiving accommodation supplement and childcare subsidy
  • Task 8: Complain about a government agency
  • Task 9: Find out who the Minister is for a specific government agency

Test data collected

A graph showing the ability of each participant in the test to complete each task, and a comparison of the time taken. See the data table below.
Figure: Number of users attempting each task, completion rates and average time on task.
Data collected for each task attempted in the user research
Task Total attempts Completed Completed on first try Average time taken (minutes)
1 11 10 8 2:56
2 11 11 11 2:39
3 11 11 10 1:48
4 11 9 1 2:51
5 10 10 7 1:56
6 10 9 8 1:06
7 8 3 3 2:22
8A 10 6 1 3:08
8B 11 5 2 2:32

Participants

Demographic profile

Eleven people participated in the two rounds of user testing:

  • 1 Female, High internet user, under 20
  • 2 Females, Medium internet user, 30-49
  • 2 Females, High internet user, 30-49
  • 1 Male, Low internet user, 50-65
  • 1 Male, High internet user, 50-65
  • 2 Males, High internet user, 20-29
  • 1 Male, Medium internet user, 20-29
  • 1 Male, High internet user, 65+

Internet usage

  • Most participants used the internet for email, banking, social networking, news and weather, watching videos (i.e. YouTube) and researching topics of personal interest.
  • Most participants used Google to find information online.

“Google sorts everything out for me.”

“Google is your friend.”

  • Four participants said that they spend more than 20 hours/week online, three said it would be between 10-20 hours/week and three said that they spend less than 10 hours/week.

Use of NZ Government sites

  • Most participants used the DIA website when getting a new passport, Council websites, the IRD website and NZQA.
  • Some also used the NZTA, StudyLink, Ministry of Justice and Statistics New Zealand websites.
  • Most seemed to use government website quite frequently, but two participants said that they don’t really use government websites.
  • Most participants (10) used government websites for private reasons and only five said that they used them work or study.

Awareness of newzealand.govt.nz

  • Six of the participants had never heard about the newzealand.govt.nz website and two participants couldn’t remember if they had heard about it.
  • Three participants said that they had heard about it and even used it, but couldn’t remember what for.
  • Most participants thought that the website would have something to do with government departments or be an
  • umbrella website for government services.

“Government departments I guess…”

  • Two participants thought that it would be a tourism or marketing website.

Preferences for finding Government information or services

  • Most participants said that they would use Google to find out more about government information or services.
  • Some said they would use the phone.

Detailed Findings

General findings

A screenshot of the homepage. Issues highlighted include the category headings, the site logo, the font being used, and the plain simple design using a grid layout.
The beta.govt.nz homepage with a number of issues highlighted, these are described in the following section of the report.

1 — Keywords in category headings (Major)

Most participants relied heavily on keywords to find the information they were looking for and struggled if they couldn’t see those words.

In one example, participants selected the ‘Identity & personal information’ section to change their driver licence details. In another example, a participant didn’t select the ‘School and college’ section to find school holiday dates, because the description didn’t include the word ‘holiday’.

Recommendation

Include popular and frequently used keywords in category headings and category descriptions.

What we think

As we add more content to the site this is something we will make adjustments to as we go. We will also monitor the search keywords people are using as this may reveal problems areas with content they can't find. We will also keep an eye on the feedback and questions we get from users.

2 — Visual design (Minor)

Most participants thought the website was plain and not attractive. But some thought that this was not necessarily a bad
thing because it didn’t distract them from finding what they were looking for.

“It doesn’t look very exciting and the colour combo is not exciting, but it looks like it has what I want.”

“It’s bland; well its black and white isn’t it.”

“It originally looked boring but as we started using it surprised me.”

Recommendation

Consider making the website more attractive without adding too much noise.

What we're doing

We've already made some updates to the site creating a new homepage, adjusting some of the fonts and colours we're using. There are more changes coming, but we're still working on these, it will be a few weeks until we're ready to launch and test them.

3 — Font (Positive)

Two participants mentioned that they liked the font.

“Font nice and easy to read, getting to my age nice to see taken into consideration.”

4 — Unfamiliar logo (Minor)

One participant said he wouldn’t trust the site because the logo was unfamiliar.

Recommendation

Consider including well-known government logos.

What we think

This recommendation contradicts our earlier research. The recommendation made here is based on the comment of a single user reacting to the New Zealand Government 'swish' logo. In earlier research we asked hundreds of users which logos and branding elements they preferred. We showed them a silver fern, the NZ Coat of Arms and the current New Zealand Government 'swish' logo. The majority of users responded saying the 'swish' was something they recognised, and the Coat of Arms was too formal and a 'bit stuffy'.

Screenshot of the Google Search homepage. The keywords users entered were recorded as part of the study.
Screenshot of the Google Search homepage. We recorded the keywords people used when searching and asked them about their familiarity with other government sites depending on what they selected in search results.

1 — Keywords (Observation)

Participants looking for an answer to Task 1 (Low income & housing support) used the search terms

  • “work and income”,
  • “wins”,
  • “work” ,
  • “accommodation”,
  • “part time employment”,
  • “new Zealand benefits”,
  • “studylink”,
  • “consumer rents” and
  • “nz.govt benefits”.

Participants looking for an answer to Task 2 (Jury duty) used the search terms

  • “jury”,
  • “jury service”,
  • “jury duty”,
  • “jury rules excuse” and
  • “nz.govt”.

2 — Familiarity with other sites (Observation)

Eight participants looking for an answer to Task 1 (Low income & housing support) said that they would go straight to the WINZ website.

“I would think to go to WINZ actually.”

Four participants looking for an answer to Task 2 (Jury duty) said that they would go straight to the Ministry of Justice website.

Homepage

The homepage of beta.govt.nz
The homepage had both positive and negative findings, including problems with navigation. Some people didn't like the categories, others found them easy to follow.

1 — Navigating to homepage (Minor)

Two participants had trouble getting back to the homepage using only the browser back button.

Most participants didn’t seem to be aware that they could use the breadcrumb trail to navigate the site.

“You lose confidence when you don’t use computers that much. I used to work on them all the time and it change so much.”

2 — Using homepage as part of task completion (Observation)

Three participants returned to the homepage before looking for a new topic.

“I return to the homepage to ground myself.”

3 — Homepage layout (Positive)

Four participants thought the homepage was clear and well laid out.

“It is quite clear the headings are good with the descriptions of what it covers underneath.”

“How newspapers used to be.”

“Like an inventory telling what the government provides.”

4 — Homepage layout (Major)

Four participants thought that the homepage was too cluttered and overwhelming.

Some participants only found what they were looking for after scrolling and scanning the page repeatedly or reading all category headings.

“Homepage a little overwhelming, whoa too much.”

“I don’t want to read all that.”

“If I’d seen it first It would have been really easy.”

Homepage (Information hub Level 1)

The homepage of beta.govt.nz showing the top level categories used to group content on the site
There were 17 different top level groupings, users relied on a combination of the main headings and the description underneath to find their way around the homepage.

1 — Consumer rights (Positive)

Ten participants who completed Task 3 (Consumer rights) went straight to the ‘Consumer rights’ section.

2 — Keywords in descriptions (Observation)

Nine participants who attempted Task 4 (Contact phone number of Ministry of Fisheries) went to the ‘Community, arts
and sport’ section.

Most based their decision to choose this category on the keyword ‘fishing’ in the category description.

3 — Alternative pathways (Observation)

Four participants who attempted Task 4 went to the ‘Environment and climate’ section.

4 — Driving and transport (Positive)

Eight participants who attempted Task 5 (Change name on driver licence) went to the ‘Driving and transport’ section.

5 — Alternative pathways (Observation)

Three participants who attempted Task 5 went to the ‘Identity and personal information’ section.

6 — Education and training (Observation)

Nine participants who attempted Task 6 (School holidays) went to the ‘Education and training’ section.

7 — Pathways not clear (Major)

Five out of eight participants who attempted Task 7 (Travelling overseas while receiving benefits) went straight to the ‘Money, benefits and tax’ section.

Four participants went to the ‘Travel and immigration’ section but two of those didn’t think they would find the information they were looking for in this section.

Recommendation

Provide multiple pathways to find information.

8 — Keywords not in description (Major)

Only three participants who attempted Task 8 (Complaint) went to the ‘Government and politics’ section.

Recommendation

Consider including the keyword ‘Complain’ in the section description.

The homepage of beta.govt.nz showing the search box in the top right corner of the screen
A lot of users we tested said they prefer search as it's a quick way of finding things. A number of people didn't see the search box.

1 — Local search (Observation)

Four participants searched for “student loan” in order to complain about the process of getting a student loan.

2 — Scrolling (Minor)

One participant didn’t see the search box because he never scrolled to the top of the page.

Recommendation

Consider keeping the search bar always visible.

3 — Preference for search (Observation)

Half of the participants said that they would use the site search because it was a quick way to find things.

“That’s what it is there for.”

“Yes, because it is quicker if it is not on the front page.”

The other half said that they wouldn’t use it because it was a waste of time and they would use Google if they needed to search.

“Seems to come back with the same thing. I can’t be bothered with it”

“When I search on sites I get a whole lot of stuff that I don't want — so I avoid site searches.”

“Haven’t found a need to. It’s the last thing you try, as it is usually quite easy to find your way through, but searching webpages had trouble. If I was going to search it would be through Google not the site search.”

“I don’t search because I like to find things for myself.”

Search results page

An example of a search results page
We hadn't done a lot of fine tuning of the search engine, so sometimes the results it showed were not considered useful.

1 — Quality of search results (Major)

Some participants seemed to expect that the search would work like Google and entered more than one search criteria, e.g. ‘student loan problem’ or search statements.

In general, too many search criteria resulted in an unsuccessful search, returning either a ‘No results found’ message or search results that seemed out of context.

“Student loan — Noooo, why would I end up with a page that is anything to do with when somebody dies?”

Recommendation

Improve site search.

Community, arts and sport page (Information hub Level 2)

The Community, arts and sport page — it links to a number of subsections
Users didn't seem to have a problem with the grid layout on these types of pages, but we think we can improve them and make it faster to get to the most popular content.

1 — Information hubs (Positive)

None of the participants seemed to have problems with this type of page.

Government and politics page (Information hub Level 2)

The Government and politics page links to a number of subcategories
Users approached the tasks we set them in a number of ways. This task provided evidence that users need a variety of pathways to find content, and that they don't always know how to find they're looking for.

1 — Multiple pathways (Observation)

Three participants went to this section to complain about a government agency.

Most decided to go straight to the agency or website of the agency they wanted to complain about.

2 — Information hub names (Positive)

Three participants went to the ‘Government and politics’ and then the ‘Parliament’ section in order to find the Minister responsible for a government agency.

Sports, fishing and hunting page (Sub-category page)

The Sports, fishing and hunting page is a subcategory of the Community, arts and sport section of the site
Once a user navigates into a subcategory of one of the main topics on the site, the links they see are to content pages. As the site grows and more content is added we might need to add more subcategories or a third level of navigation to help keep lists of content short.

1 — Sub category pages (Positive)

None of the participants seemed to have problems with this type of page.

Fishing page (Sign post page)

An example of a 'sign post' page, this is a page of content that provides links out to other websites
Our signpost pages didn't quite work as intended. Finding contact details was really hard, they were at the bottom of the page. We've now updated the design and moved the contacts into the right column. More testing now to see if they updated layout is better.

1 — Finding contact details (Major)

Nine of the participants who attempted Task 4 (Contact Phone Number for Ministry of Fisheries) went to this page.

They struggled to find a phone number and in most cases, through the ‘Fishing limits and laws’ link, went to the Ministry for Primary Industries website and found a phone number there.

Recommendation

Consider placing contact details for the responsible agencies, i.e. website address and contact phone number at the bottom of the page and in the right hand panel.

Help paying rent and housing costs page

The 'Help paying rent and housing costs' page is another example of a signpost page that links out to other websites
Users didn't tend to rely on the breadcrumbs in the header of the page to move around, most used the back button on their browser when they needed to go back to a previous page. The clear titles and strong task labels on links helped users, but like other pages, finding contact details on this page was difficult.

This page is one of the correct answers to Task 1 (Low income & housing
support).

1 — Back button (Observation)

Most participants used the browser back button to navigate the site or to return to the homepage.

2 — Breadcrumbs (Observation)

Only four participants used the breadcrumb trail to navigate the site or return to the homepage.

3 — Meaningful page titles (Observation)

All ten participants who completed Task 1 went to this page.

4 — Finding contact details (Major)

Three participants who came to this page during Task 1 weren’t sure if this was the right page or all the information they needed and wanted to call someone for more answers.

Two of those couldn’t find at phone number on this page.

“I’d call Work and Income… as I am at a standstill.”

Recommendation

Consider placing contact information at the top half of the page, so users don’t have to scroll or go to another page to find it.

5 — Calls to action (Observation)

Four participants clicked the ‘Check what WINZ benefits you might get’ link to get more information.

Budgeting and managing your money page

The 'Budgeting and managing your money' page groups links into themes and provides context for users
When asked what they would do if they were having problems paying their rent some users said they would work their way through the budgeting page. This page needs to link to the page that has information about rent and housing costs too.

This page is one of the correct answers to Task 1 (Low income & housing support).

1 — Alternative pathways (Observation)

Two of the participants who completed Task 1 went to this page.

Jury service page

The 'Jury service' page is an example of how beta.govt.nz uses a simple design to make content easy to read
Users said that the slightly larger font, lots of whitespace and plain English content helped them understand the page quickly and gave them confidence when looking for information.

This page is one of the correct answers to Task 2 (Jury duty).

1 — Content easy to understand (Positive)

All participants found the answer to Task 2 on this page and found the information easy to understand.

Your rights even if you don’t have a warranty page

A screenshot of a page that describes what you can do if you don't have a warranty
Users thought this page was easy to follow. One user commented that they didn't trust this page at first, as it was too simple. We're totally OK with a page being simple.

This page is one of the correct answers to Task 3 (Consumer rights).

1 — Content easy to understand (Positive)

Six out of eleven participants found the answer to Task 3 on this page and found the information easy to understand.

“You’d do what it says and contact the seller first and then perhaps go to disputes tribunal.”

2 — Content “too simple” (Observation)

One participant said that she wouldn’t trust this information because it seemed too simple and she would go to the Ministry for Consumer Affairs website for confirmation.

“So all this information is what I’d need to confirm that the other site is correct.”

Complain about faulty goods page

The 'Complain about faulty goods' page describes a step by step process users should take when resolving problems
This page sets out a number of things someone can do if they have a problem with faulty goods. Some users didn't notice the different steps and went straight to the first link on the page. We think we can make this clearer.

This page is one of the correct answers to Task 3 (Consumer rights).

1 — Content easy to understand (Positive)

Five out of eleven participants found the answer to Task 3 on this page and found the information easy to understand.

2 — Calls to action (Observation)

Five participants said they would go to the Disputes Tribunal.

“I’d go to the disputes tribunal website and take it from there.”

Change your name on your driver licence page

Screenshot of the page that explains how to change the name on your driver licence
This page demonstrated a classic example of what we call 'gov speak'. As far as the user is concerned, they are changing the details on their licence, but government makes them fill out a form for a 'replacement' licence. This confused a lot of people, and they commented saying things like, "But I haven't lost my licence?".

This page is one of the correct answers to Task 5 (Change name on driver licence).

1 — Content easy to understand (Positive)

Most participants found this page quickly and the content easy to understand.

“So this is easy it was clear and easy to see what you had to do.”

“That tells you what you have to take.”

2 — “Gov speak” confuses users (Minor)

Two participants were confused by the term “replacement driver licence”.

“Replacement of driving license. I don’t want a replacement I just want to change the name…”

“I’d probably think I’ve come to the wrong area. Because it’s asking me to renew my licence.”

Recommendation

Consider providing more information about why users need to replace their licence if they want to change their name.

Before you travel page

A screenshot of the 'Before you travel' page
Even though we put this page in the travel section of the site, a lot of people really didn't have any idea about what they needed to do to prepare for their trip. This page acted a bit like a checklist and helped users find content on a wide range of government sites. They still had to go to multiple places to sort everything out, but the checklist page helped them work out the most common things they'd need to think about.

This page is one of the correct answers to Task 7 (Travel overseas when receiving benefits).

1 — Preferences to call or visit (Observation)

Four participants said they would go straight to or call WINZ.

2 — Links on signpost pages (Positive)

Two participants went to this page. They selected the ‘Benefits’ link to find more information.

School terms 2014 page

The 'School terms 2014' is an example of how beta.govt.nz doesn't use tables in content
Users said this page was easy to read, and it didn't matter if they were on a desktop computer, a tablet or a smartphone. Instead of using tables which can be hard to read, and create problems for users with small screen devices, this page just lists the dates in simple bulleted lists.

This page is one of the correct answers to Task 6 (School holidays).

1 — Content easy to understand (Positive)

Most participants found this page quickly and easy to understand.

2 — Calls to action (Observation)

Two participants wanted to look up their school to get the full answer.

Contact government aka Government directory page

A screenshot of the 'Contact government' section of the site
This page enables users to filter the list of government departments and other organisations to find contact details. The visual design of the page wasn't quite right, users didn't understand what the filter was doing thinking it was actually a search box. Most users don't actually know what departments are called, and department names change reasonably frequently. In the next design update we're going to try and make this a little more obvious for people.

1 — Visual design (Minor)

Three participants didn’t see the filter option.

“No, you focus on the list.”

Recommendation

Consider making the filter option more obvious.

What we're doing

We've already starting working on some changes for this feature. They will be released to the beta site in the near future ready for the next round of testing with users.

2 — Some sections hard to find (Severe)

Most participants didn’t find this page or found it only by accident.

Recommendation

Consider making links to this section more prominent on the homepage.

What we've done

We've already updated the layout of the homepage, in the next round of testing with users we'll check to see if the new design works better or if more changes are needed.

3 — Alternative pathways (Observation)

One participant went to this page in order to complain about a government department.

4 — Identifying the right contacts (Minor)

One participant got confused when filtering by ‘Ministry of fisheries’ brought up the ‘Ministry for Primary Industries’ page.

“I’ve done something wrong.”

Recommendation

Consider including government agencies that no longer exist in the list.

What we think

Rather than adding non-existing agencies to the list of departments, we're going to make it more obvious what the new names of departments are. We were already showing people which agency they should be looking for, eg, instead of Ministry of Fisheries you need to go to the Ministry of Primary Industries now, but the design didn't make this clear. In the next version of the site design we've done some changes to make this clearer. If a user searches for an old department name, we'll show them information for the new department and say (previously known as ...).

5 — Filtering long list useful (Positive)

Participants liked this page and the filter option and found it very useful and quick.

6 — Filter vs keyword search? (Major)

Some participants thought the filter was a search and that they could enter a subject, e.g. ‘student loan’ and get the agency responsible for it.

Recommendation

Consider enabling uses to search for an agency by subject or responsibility.

Ministry for Primary Industries page (agency page)

A screenshot of an example entry in the 'Government directory' that shows contact details for the Ministry for Primary Industries
It took a while for some users to find the contact details on these pages. This means we've still got some design issues to work on. Once they had figured out that's where the phone numbers were shown, they had no problems finding the contact details on other pages they visited. The consistency of the layout is what made it easier for people. Users have told us repeatedly that finding contact details on other government websites is a major problem and really frustrating. We're going to keep working on this.

1 — Finding contact details (Major)

Four participants didn’t seem to notice the information in the right hand panel.

This was particularly problematic when they were looking for contact information or a link to an agency website, e.g. Studylink and couldn’t find it.

“Huh — ok, I feel like I have been following the wrong pathway I need contacts.”

Recommendations

Consider making the information in the right panel stand out more.

Consider keeping the content in the right hand panel always visible.

2 — Names of agencies not familiar (Observation)

Most were not surprised to end up on the Ministry for Primary Industries page or website when looking for information on the Ministry for Fisheries.

Four participants didn’t seem to know that the Ministry of Fisheries was now part of the Ministry for Primary Industries.

Browse.govt.nz page

A screenshot of the 'All categories' page on beta.govt.nz
Most users didn't find this page when they were completing the tasks we set them, but when it was shown to them most of the test participants thought this page was actually really good as it showed them all the major sections of the site on one page. We're going to make this page easier to find and compare how well it performs compared to the categories we show on the homepage of the site.

1 — Unaware of site features (Major)

Most participants didn’t notice or use the link to this page.

“I’d use this if I knew it was there.”

Recommendations

Consider making this option more prominent on the homepage.

Consider providing a link outside the footer navigation.

2 — Clearer navigation choices (Positive)

Five participants liked this page better than the homepage and said that they would prefer it to the homepage.

“I like that better as it has more details of what’s going on under the pages.”

“It is a lot nicer because it’s broken down more.”

“Awesome. Has links to things covered by that.”

3 — Filtering long list useful (Positive)

Two participants mentioned that they liked the quick filter option.

Responsible agency’ aka ‘Information provided by’ section

A screenshot of the bottom section of a content page showing how we used to link to contact details
We've tried a couple of labels for this section of the page, and neither of them really worked all that well. So it's back to basics. In the next version of the design this section will be moved to the top of the right column on the page and the heading will just be "Contacts". This should be simpler and clearer for users.

1 — Labels not clear (Minor)

Three participants expected that clicking the link under the ‘Responsible agencies’ heading would take them to the agency website and were surprised when it didn’t.

For the second round of testing the section heading was changed to “Information provided by’ and none of the participants in Round 2 of testing seemed to have this issue.

Recommendation

Consider keeping the label ‘Information provided by’.

2 — Contact details hard to find (Major)

Most participants tried to find a link to go to the agency website and were confused when they couldn’t find an obvious link at the bottom of the page or in the right hand panel.

In the second round of testing when the ‘Responsible agency’ section was changed to ‘Information provided by’, most participants seemed to ignore this section.

Recommendation

Consider keeping the label ‘Information provided by’ but placing contact details, i.e. website address and contact phone number at the bottom of the page.

3 — Navigating lists (Minor)

Seven participants assumed that the Department of Conservation was the agency responsible for fish catch limits.

Some commented that they selected this agency because it was the first one in the list.

Social media icons and footer navigation

A screenshot of the bottom section of a content page showing links to email, Twitter, Facebook and Google+
The intent of the social media sharing links was to allow users to easily share the information they found on beta.govt.nz with others. No one we tested thought these buttons were of any use, some raised concerns around privacy and 'being forced to sign up for things'. We decided to simplify the page layout and remove this feature — for now.

1 — Visual design (Observation)

Nobody seemed to miss the header and left hand panel navigation.

2 — Scrolling (Minor)

Most participants didn’t scroll all the way to the bottom on long pages and didn’t see the footer.

“Oh, this is actually registering me?”

3 — Links in footer (Positive)

Once they found it, some participants used the footer frequently because they found it easier to get an overview for what was on the site.

“I could not see it in the [page] it was there, it was the one that looked obvious.”

“I’m lazy and I’m looking for the quickest way.”

4 — Unwanted features (Observation)

Some participants noticed the social media icons and some didn’t, but most said they wouldn’t use them.

“No, only if I have too.”

“I’m worried about my privacy.”

5 — Purpose of features not clear (Minor)

Three participants thought that they could use these icons to contact a government agency.

Consider specifying ‘who’ people can contact when using these icons.

6 — Some links not obvious (Major)

Most participants didn’t realise that the headings were links.

Recommendations

Consider highlighting links on mouse over.

Consider displaying all text links in the same colour and in a colour that is different to static text.


Date of report:
18 November 2013
Version:
1.0
Prepared for:
Department of Internal Affairs
Prepared by:
Nicole Kaufmann
Email:
nicole@optimalexperience.com
Phone:
0800 872-253
Mobile:
021 460 850
Address:
Optimal Experience Limited
Level 1, 126 Cuba Street
PO Box 24032
Wellington 6011
New Zealand
Website:
www.optimalexperience.com