Data Engineer

Summary

Join Xebia as a Data Engineer and collaborate with engineering, product, and data teams to deliver scalable and robust data solutions for clients. You will design, build, and maintain data platforms and pipelines, mentor junior engineers, and ensure seamless data flow. Responsibilities include developing and maintaining data pipelines, collaborating on best practices, ensuring data integrity, integrating data from various sources, implementing ETL processes, and optimizing data storage. You will also work with cross-functional teams, monitor and troubleshoot pipelines, implement data governance policies, and develop documentation. This role requires 7+ years of experience in data engineering, proficiency with GCP services and Apache Airflow, strong Python skills, and experience with ETL tools and dbt. A Bachelor's or Master's degree in a related field is required, along with a work permit within the European Union.

Requirements

• 7+ years in a data engineering role, with hands-on experience in building data processing pipelines
• Experience in leading the design and implementing of data pipelines and data products
• Proficiency with GCP services, for large-scale data processing and optimization
• Extensive experience with Apache Airflow, including DAG creation, triggers, and workflow optimization
• Knowledge of data partitioning, batch configuration, and performance tuning for terabyte-scale processing
• Strong Python proficiency, with expertise in modern data libraries and frameworks (e.g., Databricks, Snowflake, Spark, SQL)
• Hands-on experience with ETL tools and processes
• Practical experience with dbt for data transformation
• Deep understanding of relational and NoSQL databases, data modelling, and data warehousing concepts
• Excellent command of oral and written English
• Bachelor's or Master’s degree in Computer Science, Information Systems, or a related field
• Work from the European Union region and a work permit are required

Responsibilities

• Developing and maintaining data pipelines to ensure seamless data flow from the Loyalty system to the data lake and data warehouse
• Collaborating with data engineers to ensure data engineering best practices are integrated into the development process
• Ensuring data integrity, consistency, and availability across all data systems
• Integrating data from various sources, including transactional databases, third-party APIs, and external data sources, into the data lake
• Implementing ETL processes to transform and load data into the data warehouse for analytics and reporting
• Working closely with cross-functional teams including Engineering, Business Analytics, Data Science and Product Management to understand data requirements and deliver solutions
• Collaborating with data engineers to ensure data engineering best practices are integrated into the development process
• Optimizing data storage and retrieval to improve performance and scalability
• Monitoring and troubleshooting data pipelines to ensure high reliability and efficiency
• Implementing and enforcing data governance policies to ensure data security, privacy, and compliance
• Developing documentation and standards for data processes and procedures

Preferred Qualifications

• Experience with ecommerce systems and their data integration
• Knowledge of data visualization tools (e.g., Tableau, Looker)
• Understanding of machine learning and data analytics
• Certification in cloud platforms (AWS Certified Data Analytics, Google Professional Data Engineer, etc.)