ONS Quality Questions Introduction

Note

This guidance is an ALPHA draft. It is in development and we are still working to ensure that it meets user needs.

Please get in touch with feedback to support the guidance by creating a GitHub Issue or emailing us.

Who is this guidance for?

This guidance provides a set of questions to help analytical and statistical teams evaluate the quality of their analysis throughout the production cycle.

The guidance is here to support teams in meeting the Office for National Statistics’s (ONS) strategic objectives for improving statistical quality. You can find more information about our strategic objectives on statistical quality in the ONS Statistical Quality Improvement Strategy. ONS manages quality through a strategic risk approach.

We have made the guidance available on Github in case others wish to use the Quality Questions resource in their own work.

Aims

The guidance has five main aims:

  1. To help analysts understand the level of risk they are carrying in their analytical workflows.
  2. To ensure there is a consistent end-to-end QA approach across ONS.
  3. To make it easier to comply with good practice guidance and standards including the ONS Quality Practices, ONS Quality Standard for Analysis, the government AQUA Book and the Code of Practice for Statistics, the Analysis Function Functional Standard for Analysis and the Government Service Manual which explains how to research, document and validate user needs.
  4. To ensure there is a consistent understanding of roles and responsibilities when producing high quality analysis and statistics.
  5. To make it easier to create critical project documentation including an assumptions and decisions log, issue and decisions log, risk register and divisional Quality Improvement Plan.

Reflecting on the questions asked in the template will help you to manage your analysis risks:

  1. You will be able to document the mitigation that is in place or planned.
  2. You will know which issues and risks the project is prepared to accept and why.
  3. You can identify potential quality issues and decide how to manage and prioritise them.
  4. Having this information in once place provides a sound basis for regular reviews of assumptions, issues and risks associated with the workflow, in line with recommended good practice.

The AQuA book sets out a standard framework for managing analytical quality in HM Government. AQuA is there to make sure that our work can be trusted to inform good decision making, while the Code of Practice for Statistics sets out the principles and practices that producers of official statistics should commit to.

Two other pieces of guidance have motivated us to produce this template. One is the Analysis Function guidance on Quality Questions and Red Flags. The other is the Office for Statistics Regulation (OSR) guidance on Thinking about quality when producing statistics. Both of these provide sets of questions that analysts can use to interrogate their work and assure its quality.

Building on these resources, this guidance sets out quality questions that are relevant for each stage of analytical cycle. The quality questions are at their most effective if they are asked at the right stage. Once that stage is passed, experience suggests that it is normally difficult to go back and address the points the questions ask by retrofitting at a later stage of the analysis.