Frequently Asked Questions
This guidance is an ALPHA draft. It is in development and we are still working to ensure that it meets user needs.
Please get in touch with feedback to support the guidance by creating a GitHub Issue or emailing us.
When should I use this guidance?
If you are starting work on a new analysis project we recommend that you use this guidance right from the scoping stage. For teams who are in the middle of an analysis, you can still use the guidance and templates to think about the quality questions from the scoping and design stages and continue recording the answers up until the delivery stage.
Which analysis is in scope of the guidance?
The guidance applies to all analysis workflows, including analysis to support research, advice for other departments, ad-hoc and one-off outputs and regular outputs.
Regular outputs often fulfil at least one of the criteria for what makes a business critical model:
* Underpins essential financial and funding decisions;
* Essential to the achievement of business plan actions and priorities;
* Must be error free or else risk serious financial or legal penalties or reputational damage for the organisation.
Business critical analysis requires the highest level of scrutiny and the governance arrangements must be appropriate for the level of risk.
For ad-hoc outputs with quick turnarounds and limited time and resources, analysis still needs sufficient quality assurance to ensure it is fit for purpose. Ad-hoc outputs are often used as part of the evidence when making important policy decisions, so getting them right is very important.
We strongly encourage senior responsible owners of both regular and ad-hoc analysis to answer all questions relevant to their role.
Completing the logs makes it much easier to understand the risks that the analysis carries and will help you plan to mitigate them. Having a proper audit trail and comprehensive logs also help when:
* You need to explain why the analysis works as it does.
* You need to raise quality and resourcing issues and push back against demands that are unrealistic.
* You need to understand and communicate potential risks.
How should I identify the Senior Responsible Owner, Commissioner, Analyst and Analytical Assurer?
The AQuA Book sets out four roles responsible for the assurance of analysis:
* Commissioner
* Senior Responsible Owner (SRO) (usually leads the analyst team)
* Analysts (who usually report to the SRO) and
* Analytical Assurer.
What matters here is that each assurance role is covered in your workflow and that individuals know about and accept their responsibilities, not the name of the role.
As a project team, you should make sure that each role is in place and you understand how it will operate for you so that you have the right assurance.
The AQuA Book makes no expectations about levels of seniority or grade of each of the occupiers of the roles. It does not specify whether roles should be held by a person or could be held by a committee or other governance group (such as a senior leadership team).
The key consideration is whether or not the person or group that undertakes the assurance role have the skills and resources they need to meet the requirements of the role. How the roles are covered in an analysis workflow varies from project to project, depending on how the work is planned, managed and assured.
We suggest that if roles are taken by individuals, the SRO and commissioner should usually be at Grade 7 or above. If you are still unsure about how to allocate the roles among your team and governance groups, please email us with “analysis assurance” in the subject header and we will help you to make the decision.
What help is available to answer the quality questions?
For ONS staff, the ONS Quality Central wiki contains lots of useful guidance, templates and mandatory training to help you work through and answer the quality questions. It includes the ONS Quality Standard for Analysis.
These cross-government resources are also likely to be useful:
* Government Data Quality Hub Quality Questions and Red Flags and Government Data Quality Framework
* Office for Statistics Regulation Quality Assurance of Administrative Data toolkit and Quality and statistics: an OSR perspective.
* The Uncertainty Toolkit for Analysts in Government