5 Roles and the analytical lifecycle
This version of the AQuA Book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
The draft currently has no official status. It is a work in train and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
An analytical project can be viewed as a variation on an archetypal project defined by roles and stages.
This chapter gives an overview of the roles and the project stages in the analytical lifecycle. Chapters 6 to 9 give more explanation of each stage and the responsibilities of each role during a given stage.
5.1 Roles and responsibilities
Organisations may have their own titles for the main functional roles involved in analysis that are set out here.
Each role may be fulfilled by a team or committee of people. However, a single individual (for example, the chair of a committee) will have overall accountability for each role.
The AQuA Book defines four roles.
The commissioner (may be known as customer):
- requests the analysis and sets out their requirements
- agrees that what the analyst is going to do will satisfy the need
- accepts the analysis and assurance as fit for purpose
The analyst:
- designs the approach, including the assurance, to meet the commissioner’s requirements
- agrees the approach with the commissioner
- carries out the analysis
- carries out their own assurance
- acts on findings from the assurer
- can be a group of analysts, in which case the lead analyst is responsible
The assurer (may be known as the analytical assurer or assuring analyst):
- reviews the assurance completed by the analyst
- carries out any further validation and verification they may see as appropriate
- reports errors and areas for improvement to the analyst
- undertakes repeated reviews as required
- confirms the work has been appropriately scoped, executed, validated and verified and documented to the approver
- can be a group of assurers, in which case the leader of the group is responsible
- must be independent from the analysts
The approver (may be known as senior analyst or senior responsible officer):
- scrutinises the work of the analyst and assurer
- confirms (if necessary) to the analyst, assurer and commissioner that the work has been appropriately assured
The roles of analyst and assurer shall be distinct from each other. The analyst should carry out their own assurance but responsibility for formal assurance to the approver and commissioner lies with the assurer. In some instances, particularly for quick or simple analysis, an individual may fulfil more than one of the roles, apart from the assurer and analyst roles which shall be separate from one another in all cases.
5.2 The analytical lifecycle
Quality assurance activities should take place throughout an analytical project. An effective quality assurance process involves ongoing engagement between the commissioner and the analyst to ensure an appropriate balance is maintained between time, resource and quality, and to ensure a shared understanding of the assurance activities required and risks involved.
Figure 2 is adapted from the Government Functional Standard for Analysis. Analytical quality assurance activities should take place during every phase of the cycle and should consider proportionality, although analytical quality considerations may vary depending on project governance and the specific phase of the cycle. All projects will involve some element of every phase of the cycle, even if this is not clearly defined.
It is important that proportionality is considered and that there is transparency about the analytical decisions, process, limitations and changes made at each stage to enable effective assurance and communication. This should be clearly shown in:
- documentation of the analysis, assumptions and data
- records of the analytical decisions made
- records of the quality assurance processes and checks completed
5.2.1 Engagement and scoping
Analytical projects typically start with customer engagement although other events may trigger them. Scoping ensures that an appropriate, common understanding of the problem is defined and that expectations are aligned with what can be produced. During this phase the commissioner plays an important role in communicating the questions to be addressed and working with the analyst to ensure the requirements and scope are defined and understood.
Where analysis requires multiple cycles (for example to develop, use and update analytical models), the engagement and scoping phase may follow on from the delivery and communication phase. In these cases, engagement and scoping will concentrate on the questions to be addressed in the next stage of the analytical project.
In this phase more effort may be needed to define the requirements and scope for research, evaluation or other projects that may need to seek a wider range of perspectives or for which subsequent phases and work may be provided through a product or service.
5.2.2 Design
During the design phase, the analyst will convert the commission into an analytical plan. This will set out the assurance required and ensure the analysis is sufficient to answer the questions posed. This phase includes the communication and approval of plans. Some iteration between the commissioner and the analyst is to be expected as the analytical solution is developed and limitations understood.
For larger projects or those that require multiple cycles, the design phase may include consideration of the staging of work over the whole scope of the project as well as the work required in each stage. Analysis plans for work that is dependent on insights from earlier stages may be high-level and necessitate a return to the design phase at a later date.
5.2.3 Analysis
The analysis phase is where planned analysis is undertaken and progress and relevance are monitored. The design and plan may be amended to account for changing circumstances, emerging information or unexpected difficulties or limitations encountered. This phase also includes maintaining appropriate records of the analysis conducted, changes, decisions and assumptions made. In some cases changes, or limitations encountered, may mean the scoping or design phase will need to be revisited.
Throughout this phase traceable documentation of the assurance activities that have been undertaken shall also be produced.
In larger analytical projects, some outputs of the analysis may be completed at different times as work develops and aspects of other phases may therefore take place concurrently.
5.2.4 Delivery, communication and sign-off
During the delivery stage, insights and analytical assurances are communicated to the approver and the commissioner. These should be sufficiently understood for the approver and commissioner to determine whether the work has been appropriately assured and meets their requirements. Additional analysis and further assurance may be required as analytical projects frequently need further iteration or extension to satisfy the commissioner’s needs.
Work in this stage can vary considerably depending on the commission, impact, approval processes and the nature of the project. Delivery and communication activities may include producing summary packs and reports, launching dashboards or websites and presentations.
After analysis results have been determined to meet the requirements, they are formally approved for dissemination during sign-off. Sign-off includes confirming that the commission was met, documentation and evidence are captured and appropriate assurance was conducted. This approval may be phased as work develops and insights are produced.
5.3 Maintenance and continuous review
The analytical lifecycle is not a linear process. Where analysis is used on an ongoing basis all aspects of the lifecycle should be regularly updated. For example, consideration should be made as to whether:
- the inputs used remain appropriate
- the initial communication methods remain the best way to deliver the information
- any software relied on continues to be supported and up to date
- the model continues to be calibrated appropriately (this is particularly important for black box models)
Additionally, a robust version control process should be in place to ensure any changes to the analysis are appropriately assured.
5.4 Urgent analysis
Sometimes there is a need for urgent analysis that cannot follow all the steps in this guide. For example, the need for the analysis may outweigh the risk of poor quality. In this case analysts should follow the Urgent data quality assurance guidance.