This is a living document. We want our users to get as much value as possible from our analysis. Because of that, we will be updating these pages with additional outputs and insights when possible. To make sure you always access the latest insights, we recommend linking to these pages rather than sharing this analysis any other way at this stage.

Coding frequency and tools

Coding frequency

We asked respondents “In your current role, how often do you write code to complete your work objectives?”.

Show chart Show table
Coding frequency Count
Never 16
Rarely 42
Sometimes 64
Regularly 119
All the time 88
a Sample size = 329


What people are using code for

We asked respondents what data operations they carry out in their work, and whether they use code to do them. Please note we did not ask how much of each data operation is done with code, or how often.

Show chart Show table
Data operation I do some or all of this by coding I do this without coding
Analysis 274 42
Data cleaning 255 34
Data linking 218 17
Data transfer 127 48
Data visualisation 206 94
Machine learning 41 2
Modelling 111 25
a Sample size = 329


Access to and knowledge of programming languages

Given a list of programming tools, we asked respondents to answer “Yes”, “No” or “Don’t know” for the following statements;

  • This tool is available to use for my work.
  • I know how to program with this tool to a level suitable for my work.

Access to tools does not necessarily refer to official policy. Some analysts may have access to tools others cannot access within the same organisation.

Please note that capability in programming languages is self-reported here and was not objectively defined or tested.


Access to coding tools

Show chart Show table
Programming language Yes Don't Know No
C++ / C# 14 176 139
Java / Scala 20 173 136
Javascript / Typescript 41 160 128
Matlab 6 173 150
Python 205 60 64
R 292 17 20
SAS 173 83 73
SPSS 135 94 100
SQL 230 61 38
Stata 88 140 101
VBA 209 76 44
a Sample size = 329


Knowledge of coding tools

Show chart Show table
Programming language Yes Don't Know No
C++ / C# 16 13 300
Java / Scala 13 12 304
Javascript / Typescript 22 10 297
Matlab 59 14 256
Python 117 14 198
R 247 5 77
SAS 141 10 178
SPSS 156 9 164
SQL 219 7 103
Stata 55 10 264
VBA 94 13 222
a Sample size = 329


Access and knowledge gaps

Using the data presented above we calculated the number of respondents with:

  • Access to tools they do not have the capability to use (access only),
  • Access to tools they are able to use (access and knowledge)
  • Or capability to use tools they cannot access at work (knowledge only)
Show chart Show table
Programming language Access only Access and knowledge Knowledge only
C++ / C# 12 2 14
Java / Scala 16 4 9
Javascript / Typescript 25 16 6
Matlab 6 0 59
Python 112 93 24
R 57 235 12
SAS 61 112 29
SPSS 56 79 77
SQL 46 184 35
Stata 60 28 27
VBA 122 87 7
a Sample size = 329

Coding capability


Change in coding ability during current role

This question was only asked of respondents with coding experience outside of their current role. This means analysts who first learned to code in their current role are not included in the data.

Show chart Show table
Ability Change Count
Significantly worse 20
Slightly worse 41
No change 39
Slightly better 91
Significantly better 95
a Sample size = 286


Where respondents first learned to code

Respondents with coding experience outside their current role were asked when they first learned to code. This output also includes analysts who code in their current role but reported no other coding experience.

These data only show where people first learned to code. They do not show all the settings in which they had learned to code, to what extent, or how long ago.

Show chart Show table
First coding experience Count
In current role 50
In education 155
In private sector employment 13
In public sector employment 49
Self-taught 56
Other 4
a Sample size = 327


Coding practices

We asked respondents who said they currently use code in their work how often they carry out various coding practices. For more information on the practices presented below, please read our guidance on Quality Assurance of Code for Analysis and Research.

General coding practices

Show chart Show table
Percent
Question I don't understand this question Never Rarely Sometimes Regularly All the time
I use open source software when programming 5.4 15.0 9.9 14.7 19.5 35.5
My team open sources its code 8.9 46.6 15.7 14.4 7.7 6.7
I use a source code version control system e.g. Git 3.5 37.1 18.2 10.2 13.1 17.9
Code my team writes is reviewed by a colleague 0.0 5.1 12.5 25.9 30.4 26.2
I write repetitive elements in my code as functions 2.9 7.7 11.8 25.9 33.5 18.2
I unit test my code 26.5 15.0 14.1 21.7 15.3 7.3
I collect my code and supporting material into packages 10.9 46.3 16.6 14.1 8.3 3.8
I follow a standard directory structure when programming 21.1 13.1 11.2 24.0 19.8 10.9
I follow coding guidelines or style guides when programming 4.5 8.9 10.2 26.2 35.5 14.7
I write code to automatically quality assure data 2.6 14.1 18.2 37.1 18.5 9.6
My team applies the principles set out in the Aqua book when carrying out analysis as code 29.4 15.3 7.3 18.5 21.1 8.3
a Sample size = 313


Documentation

Embedded documentation is one of the components which make up a RAP minimum viable product. Documentation is important to help others be clear on how to use the product and what the code is intended to do.

Show chart Show table
Percent
Question I don't understand this question Never Rarely Sometimes Regularly All the time
Code comments 1.9 1.6 1.0 8.9 30.7 55.9
Documentation for each function or class 11.2 18.2 15.7 22.4 22.0 10.5
README files 4.8 22.0 19.2 25.9 18.8 9.3
Desk notes 13.1 12.1 11.2 23.3 27.2 13.1
Analytical Quality Assurance (AQA) logs 25.2 25.2 17.6 15.0 12.1 4.8
Data or assumptions registers 16.0 33.9 10.5 16.3 13.7 9.6
Flow charts 6.4 34.2 22.0 25.2 9.3 2.9
a Sample size = 313


Dependency management

Respondents who currently use code in their work were asked whether they use any tools for dependency management. We provided examples of tools that may be used for dependency management:

  • Requirements files, e.g. python requirements.txt or R DESCRIPTION files
  • Virtual environments (e.g. venv or renv) or virtual machines
  • Containers e.g. Docker
Show chart Show table
Use dependency management software Count
Yes 49
No 130
I don't know what dependency management is 134
a Sample size = 313


Reproducible workflow packages

Respondents were asked whether they use continuous integration technologies. As above, respondents were provided with examples of what those might be:

  • GitHub actions
  • Jenkins
  • Travis
Show chart Show table
Use reproducible workflow packages Count
Yes 6
No 189
I don't know what reproducible workflows are 118
a Sample size = 313


RAP knowledge and opinions

We asked respondents about their knowledge of and opinions on reproducible analytical pipelines (RAP). RAP refers to the use of practices from software engineering to make analysis more reproducible. These practices build on the advantages of writing analysis as code by ensuring increased quality, trust, efficiency, business continuity and knowledge management. The RAP champions are a network of analysts across government who promote and support RAP development in their departments.


Knowledge of RAP

We asked our respondents whether they had heard of RAP and what their knowledge is of their own department RAP champion.

Show chart Show table
RAP champion knowledge Count
Have not heard of RAP 22
Heard of RAP, have not heard of RAP champions 44
Heard of RAP, does not know department champion 120
Heard of RAP champions, no champion in department 4
Knows department RAP champion 110
a Sample size = 329


Show chart Show table
Line manage anyone who writes codes count
Yes 137
No 66
I don't line manage anyone 126
a Sample size = 329


Opinions on RAP

We asked respondents who had heard of RAP whether they agreed with a series of statements.

Show chart Show table
Percent
Question Strongly disagree Disagree Neutral Agree Strongly agree
I feel confident implementing RAP in my work 8.5 16.9 23.8 33.2 17.6
I feel supported to implement RAP in my work 4.9 16.6 26.4 30.9 21.2
I know where to find resources to help me implement RAP 5.9 18.2 19.2 38.1 18.6
I understand what the key components of the RAP methodology are 6.2 14.0 17.9 42.0 19.9
I think it is important to implement RAP in my work 2.0 5.2 17.6 34.9 40.4
I and/or my team are currently implementing RAP 10.7 18.6 17.6 26.4 26.7
I or my team are planning on implementing RAP in the next 12 months 8.5 13.4 23.1 27.7 27.4
a Sample size = 307


RAP scores

RAP component scores “regularly” or “all the time” to the relevant questions. For documentation, this includes both code comments and README files. For the continuous integration and dependency management components we only collected “yes”, “no” or “I don’t understand the question” responses. As such, we gave “yes” responses a score of 1. The sum total of each respondent’s scores is presented here as “RAP scores”.

A score of one for each RAP component is derived where respondents answered “regularly” or “all the time” to the relevant questions. For documentation, this includes both code comments and README files. For the continuous integration and dependency management components we only collected “yes”, “no” or “I don’t understand the question” responses. As such, we gave “yes” responses a score of 1. The sum total of each respondent’s scores is presented here as “RAP scores”. “Basic components” are the components which make up the RAP MVP. “Advanced components” are components that help improve reproducibility, but were are considered part of the minimum standard.


RAP components

Show chart Show table
Component Type Count
AQUA book guidance Basic 92
Documentation Basic 80
Peer review Basic 177
Team open source code Basic 45
Use open source software Basic 172
Version control Basic 97
Code packages Advanced 38
Continuous integration Advanced 46
Dependency management Advanced 49
Follow code style guidelines Advanced 157
Function documentation Advanced 102
Functions Advanced 162
Unit testing Advanced 71
a Sample size = 313


Basic RAP scores

Show chart Show table
Basic RAP score Count
0 47
1 76
2 82
3 46
4 36
5 15
6 11
a Sample size = 313


Advanced RAP scores

Show chart Show table
Advanced RAP score Count
0 71
1 76
2 64
3 38
4 37
5 13
6 4
7 10
a Sample size = 313