Senior Data Engineer

Summary

Join Xebia, a global leader in digital solutions, and become a key member of our team. You will be responsible for designing, building, and deploying at-scale infrastructure, focusing on distributed systems and utilizing Big Data and Cloud technologies. Your expertise in data engineering and software development will be crucial in building and maintaining architecture patterns for data processing and system integrations. You will translate technical designs into workable solutions, create reusable artifacts, and establish automated processes for data analysis. Collaboration with analysts and data scientists is essential, as is writing efficient software and promoting good engineering practices. Excellent communication skills are required, along with experience with AWS and various data engineering tools.

Requirements

• 3+ years’ experience with AWS (Glue, Lambda, Redshift, RDS, S3)
• 5+ years’ experience with data engineering or backend/fullstack software development
• Strong SQL skills
• Python scripting proficiency
• Experience with data transformation tools - Databricks and Spark
• Data manipulation libraries (such as Pandas, NumPy, PySpark)
• Experience in structuring and modelling data in both relational and non-relational forms
• Ability to elaborate and propose relational/non-relational approach
• Normalization / denormalization and data warehousing concepts (star, snowflake schemas)
• Designing for transactional and analytical operations
• Good verbal and written communication skills in English
• Work from European Union region and work permit are required
• Candidates must have an active VAT status in the EU VIES registry: https://ec.europa.eu/taxation_customs/vies

Responsibilities

• Be responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems
• Build and maintain architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies
• Evaluate and translate technical design to workable technical solutions/code and technical specifications at par with industry standards
• Drive creation of re-usable artifacts
• Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation
• Work closely with analysts/data scientists to understand impact to the downstream data models
• Write efficient and well-organized software to ship products in an iterative, continual release environment
• Contribute and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences
• Define data retention policies
• Monitor performance and advise any necessary infrastructure changes

Preferred Qualifications

• Experience with Amazon EMR and Apache Hadoop
• Experience with data modelling tools, preferably DBT
• Experience with Enterprise Data Warehouse solutions, preferably Snowflake
• Familiarity with ETL tools (such as Informatica, Talend, Datastage, Stitch, Fivetran etc.)
• Experience in containerization and orchestration (Docker, Kubernetes etc.)
• Cloud (Azure, AWS, GCP) certification