Responding to CARS is voluntary. The results presented here are from a self-selecting sample of government analysts. Because respondents are self-selecting, the results we present reflect the views of the analysts who participated.
For more detail, see the data collection page.
We asked respondents “In your current role, how often do you write code to complete your work objectives?”
| Coding frequency | Percent |
|---|---|
| Never | 16% |
| Rarely | 4% |
| Sometimes | 28% |
| Regularly | 16% |
| All the time | 36% |
| Sample size = 25 |
Given a list of programming tools, we asked all respondents if the tool was available to use for their work.
Access to tools does not necessarily refer to official policy. Some analysts may have access to tools others cannot access within the same organisation.
| Programming tool | Yes | No | Don't Know |
|---|---|---|---|
| Python | 88% | 4% | 8% |
| R | 88% | 8% | 4% |
| SQL | 80% | 8% | 12% |
| Matlab | 0% | 48% | 52% |
| SAS | 0% | 48% | 52% |
| SPSS | 0% | 60% | 40% |
| Stata | 0% | 52% | 48% |
| VBA | 72% | 8% | 20% |
| Sample size = 25 |
Given the same list of programming tools, all respondents were asked if they knew how to program with the tool to a level suitable for their work, answering “Yes”, “No” or “Not required for my work”.
Please note that capability in programming languages is self-reported here and was not objectively defined or tested. The statement “not required for my work” was similarly not defined.
| Programming tool | Yes | No | Not required for my work |
|---|---|---|---|
| Python | 48% | 28% | 24% |
| R | 40% | 36% | 24% |
| SQL | 64% | 12% | 24% |
| Matlab | 4% | 32% | 64% |
| SAS | 8% | 32% | 60% |
| SPSS | 12% | 36% | 52% |
| Stata | 0% | 36% | 64% |
| VBA | 56% | 16% | 28% |
| Sample size = 25 |
We asked respondents to answer “Yes”, “No” or “Don’t know” for the following questions:
Please note these outputs include people who do not code at work.
Respondents with coding experience outside their current role were asked where they first learned to code. Those analysts who code in their current role but reported no other coding experience, are included as having learned ‘In current role’. Those who reported first learning to code outside of a work or educational environment were categorised as ‘self-taught’ based on free-text responses.
These data only show where people first learned to code. They do not show all the settings in which they had learned to code, to what extent, or how long ago.
| Where learned | Percent |
|---|---|
| Current employment | 23.8% |
| Education | 42.9% |
| Previous private sector employment | 23.8% |
| Previous public sector employment | 0% |
| Self-taught | 9.5% |
| Other | 0% |
| Sample size = 21 |
We asked “Has your coding ability changed during your current role?”
This question was only asked of respondents with coding experience outside of their current role. This means analysts who first learned to code in their current role are not included in the data.
| Ability change | Percent |
|---|---|
| Significantly worse | 0% |
| Slightly worse | 18.8% |
| Stayed the same | 12.5% |
| Slightly better | 18.8% |
| Significantly better | 50% |
| Sample size = 16 |
We asked respondents who said they currently use code in their work, how often they carry out various coding practices. For more information on the practices presented below, please read our guidance on Quality Assurance of Code for Analysis and Research
Open sourcing was defined as ‘making code freely available to be modified and redistributed’
| Statement | I don't understand this question (%) | Never (%) | Rarely (%) | Sometimes (%) | Regularly (%) | All the time (%) |
|---|---|---|---|---|---|---|
| Automated data quality assurance | 4.8% | 19% | 9.5% | 33.3% | 23.8% | 9.5% |
| Code review | 0% | 9.5% | 4.8% | 23.8% | 47.6% | 14.3% |
| Coding guidelines / Style guides | 0% | 19% | 4.8% | 9.5% | 57.1% | 9.5% |
| Functions | 0% | 9.5% | 4.8% | 14.3% | 33.3% | 38.1% |
| Open source own code | 4.8% | 38.1% | 23.8% | 19% | 14.3% | 0% |
| Packaging code | 0% | 33.3% | 14.3% | 38.1% | 4.8% | 9.5% |
| Proportionate quality assurance | 4.8% | 9.5% | 4.8% | 9.5% | 33.3% | 38.1% |
| Quality assurance throughout development | 0% | 9.5% | 0% | 19% | 38.1% | 33.3% |
| Standard directory structure | 28.6% | 14.3% | 4.8% | 28.6% | 9.5% | 14.3% |
| Unit testing | 23.8% | 19% | 9.5% | 23.8% | 14.3% | 9.5% |
| Use open source software | 0% | 4.8% | 4.8% | 14.3% | 28.6% | 47.6% |
| Version control | 4.8% | 23.8% | 19% | 14.3% | 9.5% | 28.6% |
| Sample size = 21 |
We asked respondents who reported writing code at work how frequently they write different forms of documentation when programming in their current role.
Embedded documentation is one of the components which make up a RAP minimum viable product. Documentation is important to help others be clear on how to use the product and what the code is intended to do.
| Statement | I don't understand this question (%) | Never (%) | Rarely (%) | Sometimes (%) | Regularly (%) | All the time (%) |
|---|---|---|---|---|---|---|
| Analytical Quality Assurance (AQA) logs | 19% | 38.1% | 0% | 28.6% | 9.5% | 4.8% |
| Code comments | 0% | 4.8% | 0% | 19% | 28.6% | 47.6% |
| Data or assumptions registers | 9.5% | 42.9% | 4.8% | 23.8% | 14.3% | 4.8% |
| Desk notes | 19% | 14.3% | 9.5% | 19% | 33.3% | 4.8% |
| Documentation for each function or class | 0% | 19% | 33.3% | 19% | 14.3% | 14.3% |
| Flow charts | 0% | 19% | 19% | 23.8% | 23.8% | 14.3% |
| README files | 0% | 38.1% | 19% | 19% | 14.3% | 9.5% |
| Sample size = 21 |
We asked respondents who reported writing code at work if they manage dependencies for their projects.
We provided examples of tools that may be used for dependency management:
| Use dependency management software | Percent |
|---|---|
| Yes | 33.3% |
| No | 38.1% |
| I don't know what dependency management is | 28.6% |
| Sample size = 21 |
We asked respondents who reported writing code at work if they use continuous integration.
We provided some examples of continuous integration technologies:
| Use continuous integration | Percent |
|---|---|
| Yes | 14.3% |
| No | 52.4% |
| I don't know what continuous integration is | 33.3% |
| Sample size = 21 |
We asked respondents who reported writing code at work whether they use reproducible workflow packages.
We provided some examples of packages:
| Use reproducible workflow packages | Percent |
|---|---|
| Yes | 14.3% |
| No | 52.4% |
| I don't know what reproducible workflows are | 33.3% |
| Sample size = 21 |
We asked respondents about their knowledge of and opinions on reproducible analytical pipelines (RAP). RAP refers to the use of practices from software engineering to make analysis more reproducible. These practices build on the advantages of writing analysis as code by ensuring increased quality, trust, efficiency, business continuity and knowledge management.
The RAP champions are a network of analysts across government who promote and support RAP development in their departments. Please contact the analysis standards and pipelines team for any enquiries about RAP or the champions network.
The Analysis Function RAP strategy was released in June 2022 and sets out plans for adopting RAP across government.
We asked respondents who reported writing code at work, if they had heard of RAP.
We asked respondents who had heard of RAP, if their department has a RAP champion and if they know who it is.
| Knowledge | Percent |
|---|---|
| Yes, and I am a RAP Champion | 14.3% |
| Yes, and I know who the RAP Champion is | 35.7% |
| Yes, but I don't know who the RAP Champion is | 7.1% |
| No | 0% |
| I don't know | 42.9% |
| Sample size = 14 |
We asked respondents who had heard of RAP, if they had heard of the RAP strategy.
| RAP strategy knowledge | Percent |
|---|---|
| Yes | 7.1% |
| Yes, but I haven't read it | 64.3% |
| No | 28.6% |
| Sample size = 14 |
We asked respondents who had heard of RAP whether they agreed with a series of statements.
| Statement | Strongly Disagree (%) | Disagree (%) | Neutral (%) | Agree (%) | Strongly Agree (%) |
|---|---|---|---|---|---|
| I and/or my team are currently implementing RAP | 7.1% | 21.4% | 50% | 7.1% | 14.3% |
| I feel confident implementing RAP in my work | 0% | 14.3% | 42.9% | 28.6% | 14.3% |
| I feel supported to implement RAP in my work | 7.1% | 14.3% | 50% | 28.6% | 0% |
| I know where to find resources to help me implement RAP | 7.1% | 21.4% | 35.7% | 28.6% | 7.1% |
| I or my team are planning on implementing RAP in the next 12 months | 0% | 7.1% | 42.9% | 35.7% | 14.3% |
| I think it is important to implement RAP in my work | 0% | 0% | 28.6% | 42.9% | 28.6% |
| I understand what the key components of the RAP methodology are | 7.1% | 14.3% | 14.3% | 42.9% | 21.4% |
| Sample size = 14 |
In this section we present RAP components and RAP scores.
For each RAP component a percent positive was calculated. Positive responses were recorded where an answer of “regularly” or “all the time” was given. For documentation, a positive response was recorded if both code comments and README files questions received positive responses. For the continuous integration and dependency management components, responses of “yes” were recorded as positive.
“Basic” components are the components which make up the RAP MVP. “Advanced” components are components which help improve reproducibility, but are not considered part of the minimum standard.
| RAP component | Type | Percentage of analysts who code in their work |
|---|---|---|
| Use open source software | Basic | 76.2% |
| Proportionate QA | Basic | 71.4% |
| Peer review | Basic | 61.9% |
| Version control | Basic | 38.1% |
| Documentation | Basic | 19% |
| Team open source code | Basic | 14.3% |
| Functions | Advanced | 71.4% |
| Follow code style guidelines | Advanced | 66.7% |
| Dependency management | Advanced | 33.3% |
| Function documentation | Advanced | 28.6% |
| Unit testing | Advanced | 23.8% |
| Code packages | Advanced | 14.3% |
| Continuous integration | Advanced | 14.3% |
| Sample size = 21 |