Home

Administrative data

This framework aims to help you assess the quality of administrative data for use in the production of official statistics.

Administrative data are data which have been collected during the operations of an organisation. Government produces a very large amount of administrative data, providing a valuable resource if it can be used correctly. There are legal gateways which can allow accredited and approved researchers to access administrative data for research and statistical purposes. There are certain criteria to meet to ensure this can happen, including the assurance that a person’s identity cannot be identified in the information disclosed for research and statistics.

Administrative data are generally not collected for the sole purpose of producing statistics. This can lead to challenges when using it for this reason, a summary of which can be found in David Hand’s paper, “Statistical challenges of administrative and transaction data”.

Accessibility

We are currently conducting work to review and, where possible, improve the accessibility of this guidance.

As part of this, we welcome any feedback on accessibility. If you find any problems, please contact us by emailing methods.research@ons.gov.uk. Please also get in touch if you are unable to access any part of this guidance, or require the content in a different format. We will consider your request and aim to get back to you within five working days.

Current accessibility features include:

• Font is arial size 12 minimum.

• You can navigate the framework using a keyboard – up and down arrow keys to scroll up and down, tab to highlight the different sections at the top, and enter to select them.

• Text can be read aloud to you (in Microsoft Edge browser) by right clicking and selecting “read aloud”.

• You can change colours, contrast levels, and font: in Microsoft Edge with “Immersive Reader” feature (right click and select “Immersive Reader”), in Google Chrome with style sheet (right click and select “Inspect”).

• You can zoom in up to 300% without the text spilling off the screen in Microsoft Edge’s “Immersive Reader” (right click and select “Immersive Reader”).

• There is alt text for the images when hovered over with a cursor.

Please note these features do not necessarily apply to the documents linked in the framework.

What is quality?

According to the Code of Practice for Statistics, “quality means that statistics fit their intended uses, are based on appropriate data and methods, and are not materially misleading.”

In essence, quality centres around a consideration of fitness for purpose:

• Is this data set good enough for what I want to use it for?

• Did the statistic I produce meet the needs of the people who are using it?

Quality assessments are important in all contexts, to ensure that the data and resulting outputs meet your needs. In an administrative data context however, as the data are often used for a purpose that is different from the reason for initial collection, there are some unique considerations to make.

The first step in any quality assessment is deciding what quality looks like for your scenario. What do the data need to do and have, to ensure you can produce what you need?

These decisions should not only factor in what high quality is to you and your user, but also the time and information you have, and any costs associated with conducting assessments or improving aspects of quality. Proportionality is a core principle of quality assessment (discussed more below).

Quality assessment

Quality is a complex area, and these are not simple decisions to make. There is not one single tool that provides every answer; we recommend developing a quality assessment procedure that draws together guidance from various packages of work.

This framework is one piece of the puzzle, but there are also other tools that can help you understand quality and the higher-level principles associated with quality assessment. Some of these are outlined below.

Code of Practice for Statistics (CoP):

• The CoP is based on three pillars; Trustworthiness, Quality and Value. Each one outlines principles and practices that official statistics producers should commit to.
• There are also three areas of practice that cut across multiple pillars; Collaboration, Coherence, and Transparency.
• Further guidance, resources, and case studies for each principle can be found at this link to the Code of Practice for Statistics.

Quality Assurance of Administrative Data (QAAD):

• The Office for Statistics Regulation has developed a regulatory standard consistent with the Code of Practice, for the quality assurance of administrative data that are used to produce official statistics.
• This link to the accompanying QAAD toolkit aims to help you apply the guidance in practice.
• The QAAD focusses on the quality assurance arrangements that are required for statistics compiled using administrative data to comply with the Code of Practice. Our framework takes a slightly different angle, providing general guidance on how to assess the quality of your data / output using various quality dimensions.
• You should use the QAAD as part of your quality assurance process, as it is a regulatory standard. How you use our framework is largely up to you; you may want to build it into a structured assessment of quality alongside the QAAD, or you may want to use it in a more casual, exploratory way (or anything in between!).
• The QAAD centres around four key practice areas:
  1. Operational context and admin data collection
  2. Communication with data supply partners
  3. Quality assurance principles, standards, and checks by data suppliers
  4. Producers’ quality assurance investigations and documentation
• The QAAD also emphasises the idea of proportionality, and the QAAD toolkit includes a helpful risk / profile matrix to help you determine the appropriate level of assurance to apply.

Generic Statistical Business Process Model (GSBPM):

• The GSBPM describes and defines the set of business processes involved in producing official statistics.
  
• It includes a framework with harmonised terminology to help with the statistical production processes, and includes guidance, indicators, and many other helpful tools.

Planned developments to this framework

We plan to add to this framework over time with other useful resources, tools, and guidance that we find and develop, and to better integrate with the other existing tools. Current planned developments are outlined below:

• The addition of case studies to help outline how others have applied this framework and used it to quality assess data or outputs.

• More detailed guidance on quality indicators / metrics / methods for the different dimensions.

• A link to an Error Catalogue that has been developed, outlining the different errors that can occur when using administrative data for statistical purposes, alongside proposed methods for measuring / addressing them.

• A link to a Conversation Toolkit to assist with communications with suppliers and asking questions about the quality of administrative data sources.

• A section covering the methodology / processing phase of the administrative data journey.

• A process map outlining the different tools and guidance in the administrative data quality space, and at what stage you might want to use them.