Gcp Data Engineer

Summary

Join Xebia as a Data Engineer and collaborate with engineering, product, and data teams to build scalable and robust data solutions for clients. You will design, build, and maintain data platforms and pipelines, mentor junior engineers, and ensure seamless data flow. Responsibilities include developing and maintaining data pipelines, integrating data from various sources, implementing ETL processes, and optimizing data storage. You will work with cross-functional teams and ensure data engineering best practices are followed. The role requires 5+ years of experience in data engineering, proficiency with GCP services, and expertise in various data tools and technologies.

Requirements

• 5+ years in a data engineering role, with hands-on experience in building data processing pipelines
• Proficiency with GCP services, for large-scale data processing and optimization
• Extensive experience with Apache Airflow, including DAG creation, triggers, and workflow optimization
• Knowledge of data partitioning, batch configuration, and performance tuning for terabyte-scale processing
• Strong Python proficiency, with expertise in modern data libraries and frameworks (e.g., Databricks, Snowflake, Spark, SQL)
• Hands-on experience with ETL tools and processes
• Deep understanding of relational and NoSQL databases, data modelling, and data warehousing concepts
• Excellent command of oral and written English
• Bachelor's or Master’s degree in Computer Science, Information Systems, or a related field
• Work from the European Union region and a work permit are required

Responsibilities

• Develop and maintain data pipelines to ensure seamless data flow from the Loyalty system to the data lake and data warehouse
• Ensure data integrity, consistency, and availability across all data systems
• Integrate data from various sources, including transactional databases, third-party APIs, and external data sources, into the data lake
• Implement ETL processes to transform and load data into the data warehouse for analytics and reporting
• Work closely with cross-functional teams including Engineering, Business Analytics, Data Science and Product Management to understand data requirements and deliver solutions
• Collaborate with data engineers to ensure data engineering best practices are integrated into the development process
• Optimize data storage and retrieval to improve performance and scalability
• Monitor and troubleshoot data pipelines to ensure high reliability and efficiency
• Implement and enforce data governance policies to ensure data security, privacy, and compliance
• Develop documentation and standards for data processes and procedures

Preferred Qualifications

• Experience with ecommerce systems and their data integration
• Knowledge of data visualization tools (e.g., Tableau, Looker)
• Understanding of machine learning and data analytics
• Certification in cloud platforms (AWS Certified Data Analytics, Google Professional Data Engineer, etc.)