Data Engineer

Summary

Join Methods Analytics as a Data Engineer and work remotely with occasional travel to client sites and offices in London, Sheffield, and Bristol. You will collaborate with cross-functional teams, design and implement ETL/ELT pipelines, build scalable data solutions, and develop data models. This role requires strong proficiency in SQL and Python, experience with Apache Spark and Azure, and excellent communication skills. You will contribute to a data-driven culture and enable informed decision-making. Methods Analytics offers a supportive environment, exciting projects, and various benefits.

Requirements

• Strong proficiency in SQL and Python for handling complex data problems
• Experience building and optimising ETL/ELT pipelines
• Hands-on experience with Apache Spark (PySpark or Spark SQL)
• Experience with the Azure data stack
• Knowledge of workflow orchestration tools like Apache Airflow
• Experience with containerisation technologies (Docker)
• Ability to craft efficient and performant queries
• Proficiency in dimensional modelling techniques
• Experience with CI/CD pipelines for data solutions
• Familiarity with test-driven development principles applied to data pipeline construction and validation
• Strong communication skills for translating technical concepts to non-technical audiences
• Business requirements analysis and translation into technical specifications
• UKSV (United Kingdom Security Vetting) clearance is required for this role, with Security Check (SC) as the minimum standard, either already held or with a willingness to undergo the process

Responsibilities

• Work closely with cross-functional teams, translating complex technical concepts into clear, accessible language for non-technical audiences
• Collaborate with a dynamic delivery team on innovative projects, transforming raw data into powerful insights
• Design and implement efficient ETL and ELT pipelines using modern tools such as Python, SQL, and Apache Airflow
• Build scalable data solutions leveraging cloud platforms and technologies
• Develop and maintain sophisticated data models, employing dimensional modelling techniques to support comprehensive data analysis and reporting
• Implement best practices in data governance, security, and compliance to maintain data integrity
• Ensure data quality through rigorous QA processes, continuously refining and optimising data queries
• Develop intuitive dashboards that provide actionable insights to stakeholders
• Monitor and tune solution performance to enhance reliability, speed, and functionality of data systems
• Stay ahead of industry trends, continuously enhancing your skills with the latest data engineering tools and methodologies
• Contribute to the development of the Methods Analytics Engineering Practice by participating in our internal community of practice

Preferred Qualifications

• Experience with data visualisation tools like Power BI or Apache Superset
• Experience with other cloud data platforms like AWS, GCP or Oracle
• Experience with modern unified data platforms like Databricks or Microsoft Fabric
• Familiarity with modern data lakehouse architectures
• Knowledge of legacy ETL tools like SSIS
• Experience with Kubernetes for container orchestration
• Understanding of streaming technologies (Apache Kafka, event-based architectures)
• Software engineering background with SOLID principles understanding
• Experience with data governance tools
• Experience with high-performance, large-scale data systems
• Familiarity with Agile development methodologies
• Knowledge of recent innovations in AI/ML and GenAI
• Defence or Public Sector experience
• Consultant experience
• Willingness to obtain DV clearance

Benefits

• Development access to Pluralsight and LinkedIn Learning
• Wellness 24/7 Confidential employee assistance programme
• Social - office parties, pizza Friday and commitment to charitable causes
• Time off - 25 days of annual leave a year, plus bank holidays, with the option to buy 5 extra days each year
• Volunteering - 2 paid days per year to volunteer in our local communities or within a charity organisation
• Pension Salary Exchange Scheme with 4% employer contribution and 5% employee contribution
• Discretionary Company Bonus based on company and individual performance
• Life Assurance of 4 times base salary
• Private Medical Insurance which is non-contributory (spouse and dependants included)
• Worldwide Travel Insurance which is non-contributory (spouse and dependants included)