Data Engineer

Summary

Join Our Future Health as a Data Engineer to help solve key challenges around a programme of work at industrial scale with global significance. You'll bring in-depth knowledge of NHS data and data solutions, working with participant data and building pipelines and systems to process it.

Requirements

• Highly proficient in Python
• Experience working with data lakes; experience with Spark, Databricks
• Understanding of common data transformation and storage formats, e.g. Apache Parquet
• Good understanding of cloud environments (ideally Azure), and workflow management systems (e.g. Dagster, Airflow, Prefect)
• Follow best practices like code review, clean code and unit tests
• Familiar with version control and Git/GitHub
• Understanding of containerisation (e.g. Docker) and deployment (e.g. Kubernetes)

Responsibilities

• Support the design and building of data pipelines from data providers to our primary data store and trusted research environment, using languages such as Python
• Produce logic for data transformation steps as code, which meets the requirements for our end users and builds well curated, accessible, and quality controlled data for analysis
• Keep abreast of best practice in data engineering across industry, research and Government and facilitating the adoption of standards
• Working with researchers to understand the data requirements and work with them to deliver the data needed for their projects
• Communicate to and between technical and non-technical stakeholders as well as facilitate discussions within a multidisciplinary team including scientists, software engineers, product managers and other data engineers

Benefits

• Up to £60,000 per annum basic salary
• Generous company pension package with employer contributions of up to 12%
• 30 days annual leave (plus bank holidays)
• Individual development budget
• Flexible and remote working arrangements and a lovely new office in Holborn, Central London