Web Standards risk assessment

The Web Accessibility Standard and Web Usability Standard require that New Zealand Government organisations be prepared to assess and report on their conformance with those Standards. This includes submitting a risk assessment and management plan regarding any areas of non-conformance. While mandated organisations will be provided in advance with an approved assessment methodology and report format, what follows can be taken as general guidance for all New Zealand Government organisations on how to approach the matter of risk in the context of the Web Standards, in particular, the Web Accessibility Standard.

This guidance directly supports the aims of:

Web Standards conformance and risk

Most New Zealand Government organisation websites must comply with the New Zealand Government Web Accessibility and Web Usability Standards. These Standards identify a number of requirements regarding web content, legal and policy issues, and notably web accessibility. If your organisation is mandated to implement the Standards, it is important to know how your sites do or do not conform, as the kind and degree of non-conformance may be placing your organisation at risk.

For example, if your site does not meet certain technical or accessibility requirements, some people with disabilities or those who use different devices might not be able to access your site. The inability of an individual or group of individuals to use your site because it is inaccessible could lead to negative consequences, including reduced uptake of online services and formal human rights complaints.

Similarly, if your site does not provide clear copyright and privacy statements, there may be no easy recourse when some of the site’s content is inappropriately repurposed, or when a citizen raises concerns about the circumstances in which her personal information has been used.

Internal sites

Publicly available websites that do not meet the Standards are obviously subject to risk in the public sphere. An internal website, however, can still present risks, for example, in terms of employee support and accommodation. This is particularly the case where the Web Accessibility Standard is concerned, as the Human Rights Act (1993), the legislation underpinning the accessibility requirements, applies equally within the New Zealand Government workplace as it does between the Government and the public. Additionally, the State Sector Act (1988) arguably requires the Public Service to ensure that its systems, including intranets and computer applications, are accessible to employees with disabilities. For these reasons, the Web Accessibility Standard applies also to internally facing web pages.

Non-mandated organisation websites

Even if your organisation is not mandated to comply with the Web Accessibility and Web Usability Standards, it remains important to know how each of your web products performs in terms of risk as your organisation remains subject to the Human Rights Act (1993) and the Bill of Rights Act (1990).

Assess and manage the risk

The identification, review, and management of risk form part of the regular strategic, operational, and planning activities surrounding any major project such as a website. Risk assessment and management help ensure that a project can achieve its objectives with the minimum of disruption and cost.

To understand the risks and consequences associated with your website’s level of accessibility or compliance with the Web Standards, a risk assessment and management plan should be created. This will enable your organisation to better address any risks your site presents, and mitigate any related outcomes.

Follow a process

Chances are that your organisation or branch already has a formal risk assessment and management process. While responsibility for each organisation’s implementation of the Web Accessibility and Web Usability Standards lies ultimately with its Chief Executive (CE), different teams at different levels within the organisation will be responsible for registering, reviewing, monitoring, and managing risk. This includes those risks generated by a site’s level of conformance to the Web Standards.

It is suggested that, where possible, you leverage your organisation’s standard risk management process and any available in-house expertise. Where this is not possible, you may wish to use a process like the following.

Identify risks

Failure to meet any aspect of the Web Accessibility or Web Usability Standard does not, in itself, present a risk. The risk lies in the consequences of that failure.

A “risk” is an event or condition (A) that leads to an outcome (B) that has an impact on the business (C). Without an impact on the business (C), there is no risk.

Risk statements

With respect to the Web Accessibility Standard, identify the failures and write corresponding risk statements. Obviously, in most cases it won’t make sense to write a risk statement for every single failure of the Standard that you find. For many websites, the individual failures will be many. Instead, it will make more sense to group and classify the failures based on their type and effect. You may also wish to include known issues that aren’t direct failures of the Web Accessibility Standard but may still cause a risk, such as the use of CAPTCHAs or very small font sizes.

The risk statement might look something like one of the following (fictitious) examples:

  • “The instructional videos on our site are not captioned, which means a significant community (Deaf) is not able to use our online services, causing a negative impact on their uptake by the public and giving rise to the possibility of a human rights complaint.”
  • “Licence application forms on our site fail WCAG 2.0 Success Criterion 2.1.1 and are not usable by keyboard users. Some users therefore cannot apply for a licence online, which incurs extra cost and the inconvenience of providing face-to-face service, and exposes the organisation to risk of complaint to the Human Rights Commission (HRC) on the grounds of discrimination.”
  • “Our publishing procedures result in inaccessible PDF documents without accessible alternatives, which results in requests via our 0800 number, which in turn incurs additional costs for printing and posting out hard-copy documents.”

Assess the risks

Risk severity

Assess the nature and severity of each risk on a spectrum from minimal to severe. For example, the following table provides a way to assess the severity of potential risks in different impact areas stemming from a lack of conformance with the Web Accessibility Standard.

Web Accessibility Standard risk severity assessment
Rating Severity Reputation Service Delivery Legal Fiscal
5 Severe
  • New Zealand Government embarrassed publicly
  • Adverse impact on reporting to UN on UNCRPD
Inaccessibility of the online service extends beyond disability community, and overall uptake of the service fails to meet target
  • Court proceedings for breach of NZ Bill of Rights Act (1990). Compensation may be payable
  • Judicial Review proceedings
  • Complaint to the Human Rights Review Tribunal for a breach of the Human Rights Act (1993). Damages can be awarded up to $200,000
Organisation requires additional funding from Government
4 Significant
  • Organisation seen by disability community as dismissive of need for accessibility
  • Agitation by disability groups in general media
Public dissatisfaction within disability community adversely affects overall usage of online services or information
  • Complaint to the Human Rights Commission for discrimination on the grounds of disability
  • Complaint to Ombudsman regarding UNCRPD or decision making process
  • Breach of Cabinet direction to comply with Web Standards
  • Cost of legal defence or remediation is high enough to impact organisation’s overall activities
  • Human resources and finances have to be diverted away from other programs
3 Moderate Public comment by disability community about organisation’s competence online
  • Numerically significant impact on usage of online services
  • Large disability communities unable to use online services or information
Public threat(s) of complaint to HRC, Ombudsman, or legal action against organisation Cost of remediation or dealing with complaint requires additional funding outside baseline
2 Minor
  • Public comments within one disability group
  • Questions about organisation’s online competence from one disability group
  • Measurable but minor negative impact on usage of online services
  • One disability group largely unable to move some business with government online
Formal complaints laid with organisation Cost of remediation or dealing with complaint requires adjustment within organisation baseline
1 Minimal Some complaints from dissatisfied users
  • No measurable impact on uptake of online services
  • Some users have difficulty using online services or information
Internal legal advice may be required to respond Cost of remediation can be covered as part of the business’s normal operations at the expense of other activities

Risk likelihood

Next, assuming that a particular risk’s cause has not been addressed or mitigated, what is the likelihood of the business impact of that risk manifesting? In the area of accessibility, business impacts may include:

  • the cost and resource of a response to a complaint or action against the organisation, or
  • the impact on the extent of uptake of an online service, or
  • the often hidden cost of potential online users resorting to face-to-face or call centre service if they are unable to use an online, self-service option.

Assess what impacts there may be for your organisation’s services, and then assess the likelihood of that impact becoming real.

For example, in the context of risks associated with a failure to meet the Web Accessibility Standard, the following questions could be asked to assess that risk’s likelihood:

  • How widespread is the failure? Does the failure occur in isolated instances, or is it endemic across a site?
  • Does the failure prevent or hinder access to high-stakes information or services?
  • What is the impact on users? Does the failure merely hinder or does it completely prevent access to information or services for some users?
  • What type of disability groups are most affected by the failure? Are multiple disability groups affected?
  • Is the failure one that’s already been described in the Office for Disability Issues’ list of common website accessibility barriers and solutions?

Risk rating

Rate the unmitigated risks by plotting them in a matrix like the one below that relates the severity of risk impact to the likelihood of the risk occurring. Using the matrix, you can assign each risk a rating from 1 to 25. A higher risk rating represents a greater risk and a higher priority for mitigation. Each risk rating has an associated colour to depict its general risk level: red for extreme, orange for high, yellow for moderate, and green for low).

Risk rating matrix

Table for rating risk level as the product of a risk's impact severity (left column) against its likelihood (first row).

Almost never Possible but unlikely Possible Highly probable Almost certain
Severe impact 15 (orange) 19 (orange) 22 (red) 24 (red) 25 (red)
Significant impact 10 (yellow) 14 (orange) 18 (orange) 21 (orange) 23 (red)
Moderate impact 6 (yellow) 9 (yellow) 13 (yellow) 17 (orange) 20 (orange)
Minor impact 3 (green) 5 (yellow) 8 (yellow) 12 (yellow) 16 (orange)
Minimal impact 1 (green) 2 (green) 4 (yellow) 7 (yellow) 11 (yellow)

Respond to the risks

Consider the effect of remediation or repair on each risk. What measures or controls could be applied to reduce or eliminate the level of risk, and what costs would be involved? Risk reduction can result from a reduction of either or both the risk’s severity or likelihood. Removing the risk’s original cause usually eliminates the risk entirely.

Put together a list of risks and the ways to address them. This can be used to evaluate and prepare a plan to reduce the number of risks or their impact.

Next, re-assess the severity, likelihood, and rating of each risk were it to be treated with the proposed controls and measures. Any risk that remains after treatment is ‘residual risk’. Residual risk should be accepted and signed off by an appropriate level of management.

Report the risks

Escalation

Organisations should establish an escalation path to ensure appropriate levels of management are aware of the risks that the organisation or project carries, and be in a position to decide or agree on appropriate action. It may look something like the following:

Risk escalation paths
Risk Escalation path
22-25 (red) Deputy Chief Executive or Business Owner
14-21 (orange) Programme Director
4-13 (yellow) Project Manager
1-3 (green) Project Team

Risk register

To keep track of the risks you are managing, plot each risk in a register. For each risk, describe it, its area of impact, its consequences, and any controls that could mitigate that risk. Also note the severity, likelihood, and risk rating of the risk in both its untreated and treated forms.

Below is an example of a basic risk register. Three sample issues have been entered into the template. Please note that, while certainly possible, these are fictitious issues only. In the context of a real organisation and website, the same risks might present themselves with different potential consequences, receive different assessments, and suggest different responses.

Sample risk register (example only)
Risk ID Risk description Area of impact Impact description Severity Likelihood Risk rating Controls Residual severity Residual likelihood Residual risk rating
R01 Forms in public-facing web application are not keyboard accessible: People with certain disabilities or those with older mobile devices will not be able to submit the forms, which is the main purpose of the application.
  • Reputation
  • Legal
  • Human rights complaints or legal action
  • Negative public/media exposure
  • International scrutiny and loss of reputation
Moderate Possible 13 (yellow)
  • Add keyboard accessibility support
  • Provide alternative (phone/in-person) access to forms
  • Accept risk regarding older mobile devices
N/A N/A None
R02 A few layout tables have non-empty “summary” attributes: Some screen reader users will suffer undue noise when navigating/reading table content. Reputation Limited user complaints Minor Possible but unlikely 5 (yellow)
  • Acknowledge that the site is not perfect, but note that it is not inaccessible.
  • Review opportunity to implement fixes and remove impact caused by “summary” attributes.
Minor Possible but unlikely 5 (yellow)
R03 Three two-minute videos for children on how to stay safe around dogs are not captioned and have no transcript. Deaf children or parents are deprived of important information related to their safety.
  • Reputation
  • Legal
A complaint of discrimination to the HRC is possible if requests to caption go unresolved. Significant Possible 18 (orange) Provide text transcript Minor Possible 8 (yellow)

Something is better than nothing

Where the approach described above is not possible, or where a formal risk assessment is not a requirement, you should at least take some time to review the state of your site’s adherence to the Web Standards, and based on this:

  1. identify risks related to shortcomings in your site’s accessibility and standards-compliance;
  2. assess and rate the likelihood and possible consequences of those risks; and,
  3. plan responses for each potential risk scenario, including what can be done immediately to reduce the likelihood and later impact of those risks.