Azure Data Engineer

Summary

Join Xebia, a global leader in digital solutions, and become a key member of our team. We are seeking a skilled Data Engineer with 3+ years of experience in data engineering or backend/fullstack software development and expertise in Azure (Data Factory, Databricks). You will be responsible for designing, building, and deploying at-scale infrastructure, building and maintaining architecture patterns for data processing, and working closely with analysts and data scientists. This role requires proficiency in SQL, Git, Python scripting, and experience with data transformation tools. We offer a collaborative environment focused on professional development and innovation. While we may not have an immediate project, we are proactively recruiting for future opportunities.

Requirements

• Have 2+ years’ experience with Azure (Data Factory, Databricks)
• Have 3+ years’ experience with data engineering or backend/fullstack software development
• Possess solid SQL and Git skills
• Have Python scripting proficiency
• Have experience with data transformation tools - Databricks and Spark
• Have experience in structuring and modelling data in both relational and non-relational forms
• Be able to elaborate and propose relational/non-relational approach
• Understand normalization / denormalization and data warehousing concepts (star, snowflake schemas)
• Have good verbal and written communication skills in English
• Work from the European Union region and have a work permit

Responsibilities

• Be responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems
• Build and maintain architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies
• Evaluate and translate technical design to workable technical solutions/code and technical specifications at par with industry standards
• Drive creation of re-usable artifacts
• Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation
• Work closely with analysts/data scientists to understand impact to the downstream data models
• Write efficient and well-organized software to ship products in an iterative, continual release environment
• Contribute and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences
• Define data retention policies
• Monitor performance and advise any necessary infrastructure changes

Preferred Qualifications

• Have experience with CI/CD tooling (GitHub, Azure DevOps, Harness etc.)
• Be familiar with data manipulation libraries (such as Pandas, NumPy, PySpark)
• Have experience with Azure Event Hubs, Azure Blob Storage, Azure Synapse, Spark Streaming
• Have experience with data modelling tools, preferably DBT
• Have experience with Enterprise Data Warehouse solutions, preferably Snowflake
• Be familiar with ETL tools (such as Informatica, Talend, Datastage, Stitch, Fivetran etc)
• Have experience in containerization and orchestration (Docker, Kubernetes etc)
• Have a cloud (Azure, AWS, GCP) certification