Home

Note

This is an ALPHA draft and is published for feedback purposes. The contents of these pages may change in response to feedback and suggestions.

Aims

This document sets a minimum standard for Reproducible Analytical Pipelines (RAP) for analysts at the Office for National Statistics. RAP refers to the use of tools and practices borrowed from software engineering to make our analysis more reproducible.

Analysis is reproducible if we can repeat the analysis process reliably. In practice, that means we can always produce the same outputs again given the same data. The “gold standard” for reproducibility is a pipeline that is robust and well documented enough that another analyst could reproduce your outputs with no support except your code, documentation, and data.

There is no single way to make analysis code reproducible. Rather, there are different ways of working that can help us get to that goal. We hope this document provides useful and coherent criteria for how a Reproducible Analytical Pipeline at ONS should look in practice.

Reproducibility is not an end in itself. Rather, it is necessary so our work has quality, value, and transparency. The minimum RAP standards are written with this in mind. The criteria set out in this document are there to make sure we get the most value out of reproducible analysis.

How should the standard be used?

The standard sets a minimum set of expectations that code should comply with if it is used to produce analytical outputs. The analysis might be ad hoc, or it might be used to create official and national statistics or underpin other ONS analysis.

While this is a minimum standard, there are many other techniques, tools and practices that can improve your code and make it even more efficient and reproducible. Once you comply with the minimum standard, we hope you will build on it to make your work even more resilient.

We do not expect all coding projects to meet the standard immediately. If your work does not comply with the minimum standard, you should put in place coherent, prioritised and achievable plans for how you will achieve compliance with the standard in time.

Most of the practices and activities here apply to both re-usable pipelines and ad hoc analysis. Criteria that only apply to re-usable pipelines are highlighted when they appear.

Lastly, this document is not intended to teach coding skills. To improve your coding capability you can read our guidance on good coding practices or use ONS internal training courses, in particular the RAP Learning Pathway on the ONS Learning Hub.

Accessibility statement

This accessibility statement applies to the RAP minimum standards resource. Please note that this does not include third-party content that is referenced from this site.

The website is managed by the Methodology and Quality division of the Office for National Statistics. We would like this guidance to be accessible for as many people as possible. This means that you should be able to:

  • change colours, contrast levels and fonts
  • zoom in up to 300% without the text spilling off the screen
  • navigate most of the website using just a keyboard
  • navigate most of the website using speech recognition software
  • listen to most of the website using a screen reader (including the most recent versions of JAWS, NVDA and VoiceOver)

Feedback and reporting accessibility problems

We’re always looking to improve the accessibility of our guidance. If you find any problems not listed on this page or think that we’re not meeting accessibility requirements, please contact us by email at gsshelp@statistics.gov.uk. Please also get in touch if you are unable to access any part of this guidance, or require the content in a different format.

We will consider your request and aim to get back to you within 5 working days.

Enforcement procedure

The Equality and Human Rights Commission (EHRC) is responsible for enforcing the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018 (the ‘accessibility regulations’). If you’re not happy with how we respond to your complaint, you should contact the Equality Advisory and Support Service (EASS).

Contact details

We would like to know what you think of the RAP minimum standard. Please contact us if you have feedback on any of the following:

  • How we might make the guidance clearer and easier to follow
  • Information you would like to see collected in the next iteration of the guidance
  • Any other comments

To give feedback, please contact ASAP@ONS.gov.uk.