Senior Data Engineer

Summary

Join VivaCity as a Senior Platform Engineer and play a crucial role in designing, improving, and maintaining our critical data systems. You will be responsible for our cloud data pipeline and infrastructure, working with technologies like Apache Kafka, Kubernetes, and PostgreSQL. This role requires deep expertise in data platform engineering, distributed systems, and cloud infrastructure. You will collaborate with the Cloud and AI team, establish technical standards, and mentor other engineers. We offer a flexible and hybrid working environment, along with a comprehensive benefits package.

Requirements

• Deep expertise in Apache Kafka (ideally Kafka Strimzi), including broker management, scaling, upgrades, and integration with high-throughput data pipelines
• Strong experience designing, implementing and managing cloud infrastructure with a focus on reliability, simplicity, and cost-effectiveness (we use Kubernetes on Google Cloud Platform)
• Proficiency in Unix systems, ideally Linux (we use Ubuntu)
• Exceptional communication skills, with experience mentoring engineers and collaborating with stakeholders
• Proven ability to resolve technical incidents in unfamiliar production systems
• Technical and process documentation champion
• Experience of operationally managing production software components, including observability, logging, metrics, error reporting, debugging, and live incident management

Responsibilities

• Design, improve, and maintain critical systems which transport hundreds of gigabytes per day and trillions of messages per year
• Ensure the reliability, simplicity, and cost-effectiveness of our systems
• Take primary accountability for our cloud data pipeline & underlying infrastructure, including Apache Kafka, go microservices, a PostgreSQL database, Kubernetes, Vault, and other cloud services, as well as on-premise infrastructure
• Optimize the performance, maintainability, and operational costs of these systems while ensuring they meet business needs
• Collaborate closely with the Cloud and AI team
• Establish technical standards
• Lead architectural decision-making
• Proactively share knowledge with colleagues
• Provide mentorship, technical guidance and training to engineers across the department

Preferred Qualifications

• Proficiency with Hashicorp Vault including secret and role management
• Proficiency with PostgreSQL (we use TimescaleDB), including query optimisation and SQL proficiency
• Proficiency in one or more programming languages (we primarily use Go and TypeScript)
• Comfortable working with Infrastructure as Code, ideally Terraform
• Familiarity with common DevOps tools and CI/CD processes
• Proficiency with infrastructure management tools (e.g. Ansible, Chef, Puppet)
• Experience with internal developer platforms
• Knowledge of information security principles, including identity and access management and authentication/authorisation processes
• Experience in a start-up or scale-up environment
• Passion for sustainability, transport or smart cities

Benefits

• 25 days of holiday
• Flexible working hours and work from home
• Up to 30 days per year work from abroad
• Up to 3 flexible bank holiday days
• 3% employer pension contribution
• ��200 total working from home budget
• Enhanced parental leave
• Up to 3 ‘childcare’ or ‘dependant’ days per year
• Mental health support through Spill
• Discounted gym memberships through GymFlex
• ��50 per person per quarter team social budget
• Additional entertainment budget and company parties
• Office food and drinks
• Optional Bupa healthcare and dental care through salary deduction
• Eye care vouchers
• Seasonal Flu jabs
• Cycle to work scheme
• Payroll giving through salary sacrifice