This is a living document. We want our users to get as much value as possible from our analysis. Because of that, we will be updating these pages with additional outputs and insights when possible. To make sure you always access the latest insights, we recommend linking to these pages rather than sharing this analysis any other way at this stage.
We asked respondents “In your current role, how often do you write code to complete your work objectives?”.
Show chart Show table| Coding frequency | Count |
|---|---|
| Never | 2 |
| Rarely | 2 |
| Sometimes | 2 |
| Regularly | 7 |
| All the time | 17 |
| a Sample size = 30 |
We asked respondents what data operations they carry out in their work, and whether they use code to do them. Please note we did not ask how much of each data operation is done with code, or how often.
Show chart Show table| Data operation | I do some or all of this by coding | I do this without coding |
|---|---|---|
| Analysis | 26 | 4 |
| Data cleaning | 26 | 2 |
| Data linking | 26 | 0 |
| Data transfer | 17 | 2 |
| Data visualisation | 25 | 3 |
| Machine learning | 12 | 0 |
| Modelling | 19 | 2 |
| a Sample size = 30 |
Given a list of programming tools, we asked respondents to answer “Yes”, “No” or “Don’t know” for the following statements;
Access to tools does not necessarily refer to official policy. Some analysts may have access to tools others cannot access within the same organisation.
Please note that capability in programming languages is self-reported here and was not objectively defined or tested.
| Programming language | Yes | Don't Know | No |
|---|---|---|---|
| C++ / C# | 1 | 11 | 18 |
| Java / Scala | 4 | 8 | 18 |
| Javascript / Typescript | 5 | 12 | 13 |
| Matlab | 0 | 8 | 22 |
| Python | 23 | 3 | 4 |
| R | 29 | 0 | 1 |
| SAS | 3 | 7 | 20 |
| SPSS | 3 | 6 | 21 |
| SQL | 30 | 0 | 0 |
| Stata | 1 | 8 | 21 |
| VBA | 17 | 6 | 7 |
| a Sample size = 30 |
| Programming language | Yes | Don't Know | No |
|---|---|---|---|
| C++ / C# | 3 | 3 | 24 |
| Java / Scala | 2 | 1 | 27 |
| Javascript / Typescript | 6 | 2 | 22 |
| Matlab | 5 | 3 | 22 |
| Python | 13 | 1 | 16 |
| R | 21 | 1 | 8 |
| SAS | 3 | 2 | 25 |
| SPSS | 10 | 2 | 18 |
| SQL | 26 | 1 | 3 |
| Stata | 2 | 3 | 25 |
| VBA | 15 | 0 | 15 |
| a Sample size = 30 |
Using the data presented above we calculated the number of respondents with:
| Programming language | Access only | Access and knowledge | Knowledge only |
|---|---|---|---|
| C++ / C# | 1 | 0 | 3 |
| Java / Scala | 4 | 0 | 2 |
| Javascript / Typescript | 2 | 3 | 3 |
| Matlab | 0 | 0 | 5 |
| Python | 13 | 10 | 3 |
| R | 8 | 21 | 0 |
| SAS | 2 | 1 | 2 |
| SPSS | 1 | 2 | 8 |
| SQL | 4 | 26 | 0 |
| Stata | 1 | 0 | 2 |
| VBA | 3 | 14 | 1 |
| a Sample size = 30 |
This question was only asked of respondents with coding experience outside of their current role. This means analysts who first learned to code in their current role are not included in the data.
Show chart Show table| Ability Change | Count |
|---|---|
| Significantly worse | 1 |
| Slightly worse | 1 |
| No change | 1 |
| Slightly better | 10 |
| Significantly better | 12 |
| a Sample size = 25 |
Respondents with coding experience outside their current role were asked when they first learned to code. This output also includes analysts who code in their current role but reported no other coding experience.
These data only show where people first learned to code. They do not show all the settings in which they had learned to code, to what extent, or how long ago.
Show chart Show table| First coding experience | Count |
|---|---|
| In current role | 3 |
| In education | 14 |
| In private sector employment | 3 |
| In public sector employment | 3 |
| Self-taught | 4 |
| Other | 1 |
| a Sample size = 28 |
We asked respondents who said they currently use code in their work how often they carry out various coding practices. For more information on the practices presented below, please read our guidance on Quality Assurance of Code for Analysis and Research.
| Question | I don't understand this question | Never | Rarely | Sometimes | Regularly | All the time |
|---|---|---|---|---|---|---|
| I use open source software when programming | 0.0 | 10.7 | 14.3 | 3.6 | 35.7 | 35.7 |
| My team open sources its code | 3.6 | 46.4 | 25.0 | 10.7 | 14.3 | 0.0 |
| I use a source code version control system e.g. Git | 0.0 | 42.9 | 3.6 | 10.7 | 21.4 | 21.4 |
| Code my team writes is reviewed by a colleague | 0.0 | 0.0 | 3.6 | 3.6 | 32.1 | 60.7 |
| I write repetitive elements in my code as functions | 3.6 | 3.6 | 7.1 | 14.3 | 42.9 | 28.6 |
| I unit test my code | 21.4 | 7.1 | 7.1 | 25.0 | 21.4 | 17.9 |
| I collect my code and supporting material into packages | 7.1 | 28.6 | 25.0 | 14.3 | 21.4 | 3.6 |
| I follow a standard directory structure when programming | 14.3 | 7.1 | 7.1 | 21.4 | 32.1 | 17.9 |
| I follow coding guidelines or style guides when programming | 0.0 | 3.6 | 3.6 | 21.4 | 46.4 | 25.0 |
| I write code to automatically quality assure data | 0.0 | 7.1 | 28.6 | 39.3 | 14.3 | 10.7 |
| My team applies the principles set out in the Aqua book when carrying out analysis as code | 60.7 | 17.9 | 3.6 | 3.6 | 10.7 | 3.6 |
| a Sample size = 28 |
Embedded documentation is one of the components which make up a RAP minimum viable product. Documentation is important to help others be clear on how to use the product and what the code is intended to do.
Show chart Show table| Question | I don't understand this question | Never | Rarely | Sometimes | Regularly | All the time |
|---|---|---|---|---|---|---|
| Code comments | 0.0 | 3.6 | 0.0 | 7.1 | 25.0 | 64.3 |
| Documentation for each function or class | 7.1 | 25.0 | 3.6 | 21.4 | 35.7 | 7.1 |
| README files | 7.1 | 25.0 | 14.3 | 21.4 | 21.4 | 10.7 |
| Desk notes | 17.9 | 50.0 | 14.3 | 14.3 | 3.6 | 0.0 |
| Analytical Quality Assurance (AQA) logs | 10.7 | 35.7 | 17.9 | 14.3 | 14.3 | 7.1 |
| Data or assumptions registers | 25.0 | 46.4 | 7.1 | 10.7 | 10.7 | 0.0 |
| Flow charts | 3.6 | 42.9 | 25.0 | 14.3 | 10.7 | 3.6 |
| a Sample size = 28 |
Respondents who currently use code in their work were asked whether they use any tools for dependency management. We provided examples of tools that may be used for dependency management:
| Use dependency management software | Count |
|---|---|
| Yes | 8 |
| No | 14 |
| I don't know what dependency management is | 6 |
| a Sample size = 28 |
Respondents were asked whether they use continuous integration technologies. As above, respondents were provided with examples of what those might be:
| Use reproducible workflow packages | Count |
|---|---|
| Yes | 0 |
| No | 24 |
| I don't know what reproducible workflows are | 4 |
| a Sample size = 28 |
We asked respondents about their knowledge of and opinions on reproducible analytical pipelines (RAP). RAP refers to the use of practices from software engineering to make analysis more reproducible. These practices build on the advantages of writing analysis as code by ensuring increased quality, trust, efficiency, business continuity and knowledge management. The RAP champions are a network of analysts across government who promote and support RAP development in their departments.
We asked our respondents whether they had heard of RAP and what their knowledge is of their own department RAP champion.
Show chart Show table| RAP champion knowledge | Count |
|---|---|
| Have not heard of RAP | 11 |
| Heard of RAP, have not heard of RAP champions | 7 |
| Heard of RAP, does not know department champion | 3 |
| Heard of RAP champions, no champion in department | 1 |
| Knows department RAP champion | 5 |
| a Sample size = 30 |
| Line manage anyone who writes codes | count |
|---|---|
| Yes | 9 |
| No | 7 |
| I don't line manage anyone | 14 |
| a Sample size = 30 |
We asked respondents who had heard of RAP whether they agreed with a series of statements.
Show chart Show table| Question | Strongly disagree | Disagree | Neutral | Agree | Strongly agree |
|---|---|---|---|---|---|
| I feel confident implementing RAP in my work | 5.3 | 5.3 | 31.6 | 31.6 | 26.3 |
| I feel supported to implement RAP in my work | 5.3 | 5.3 | 26.3 | 31.6 | 31.6 |
| I know where to find resources to help me implement RAP | 10.5 | 5.3 | 15.8 | 42.1 | 26.3 |
| I understand what the key components of the RAP methodology are | 5.3 | 10.5 | 26.3 | 42.1 | 15.8 |
| I think it is important to implement RAP in my work | 0.0 | 0.0 | 26.3 | 42.1 | 31.6 |
| I and/or my team are currently implementing RAP | 5.3 | 15.8 | 31.6 | 21.1 | 26.3 |
| I or my team are planning on implementing RAP in the next 12 months | 0.0 | 10.5 | 31.6 | 31.6 | 26.3 |
| a Sample size = 19 |
RAP component scores “regularly” or “all the time” to the relevant questions. For documentation, this includes both code comments and README files. For the continuous integration and dependency management components we only collected “yes”, “no” or “I don’t understand the question” responses. As such, we gave “yes” responses a score of 1. The sum total of each respondent’s scores is presented here as “RAP scores”.
A score of one for each RAP component is derived where respondents answered “regularly” or “all the time” to the relevant questions. For documentation, this includes both code comments and README files. For the continuous integration and dependency management components we only collected “yes”, “no” or “I don’t understand the question” responses. As such, we gave “yes” responses a score of 1. The sum total of each respondent’s scores is presented here as “RAP scores”. “Basic components” are the components which make up the RAP MVP. “Advanced components” are components that help improve reproducibility, but were are considered part of the minimum standard.
| Component | Type | Count |
|---|---|---|
| AQUA book guidance | Basic | 4 |
| Documentation | Basic | 8 |
| Peer review | Basic | 26 |
| Team open source code | Basic | 4 |
| Use open source software | Basic | 20 |
| Version control | Basic | 12 |
| Code packages | Advanced | 7 |
| Continuous integration | Advanced | 8 |
| Dependency management | Advanced | 8 |
| Follow code style guidelines | Advanced | 20 |
| Function documentation | Advanced | 12 |
| Functions | Advanced | 20 |
| Unit testing | Advanced | 11 |
| a Sample size = 28 |
| Basic RAP score | Count |
|---|---|
| 0 | 1 |
| 1 | 7 |
| 2 | 7 |
| 3 | 5 |
| 4 | 3 |
| 5 | 4 |
| 6 | 1 |
| a Sample size = 28 |
| Advanced RAP score | Count |
|---|---|
| 0 | 1 |
| 1 | 6 |
| 2 | 7 |
| 3 | 2 |
| 4 | 6 |
| 5 | 2 |
| 6 | 2 |
| 7 | 2 |
| a Sample size = 28 |