Azure Data Engineer

Summary

Join Xebia, a global leader in digital solutions, and contribute to cutting-edge cloud-based projects. As a Data Engineer, you will be responsible for designing, building, and deploying at-scale infrastructure, focusing on distributed systems and Big Data technologies. You will work with various cloud platforms, including Azure, and collaborate with data scientists to ensure efficient data processing and analysis. This role requires strong experience in data engineering, software development, and SQL, along with proficiency in Python and data transformation tools. Xebia offers a collaborative environment and opportunities for professional growth.

Requirements

• Have 2+ years’ experience with Azure (Data Factory, Databricks)
• Have 3+ years’ experience with data engineering or backend/fullstack software development
• Possess solid SQL and Git skills
• Have Python scripting proficiency
• Have experience with data transformation tools - Databricks and Spark
• Have experience in structuring and modelling data in both relational and non-relational forms
• Be able to elaborate and propose relational/non-relational approach
• Understand normalization / denormalization and data warehousing concepts (star, snowflake schemas)
• Have good verbal and written communication skills in English
• Work from the European Union region and have a work permit

Responsibilities

• Be responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems
• Build and maintain architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies
• Evaluate and translate technical design to workable technical solutions/code and technical specifications at par with industry standards
• Drive creation of re-usable artifacts
• Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation
• Work closely with analysts/data scientists to understand impact to the downstream data models
• Write efficient and well-organized software to ship products in an iterative, continual release environment
• Contribute and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences
• Define data retention policies
• Monitor performance and advise any necessary infrastructure changes

Preferred Qualifications

• Have experience with CI/CD tooling (GitHub, Azure DevOps, Harness etc.)
• Be familiar with data manipulation libraries (such as Pandas, NumPy, PySpark)
• Have experience with Azure Event Hubs, Azure Blob Storage, Azure Synapse, Spark Streaming
• Have experience with data modelling tools, preferably DBT
• Have experience with Enterprise Data Warehouse solutions, preferably Snowflake
• Be familiar with ETL tools (such as Informatica, Talend, Datastage, Stitch, Fivetran etc)
• Have experience in containerization and orchestration (Docker, Kubernetes etc)
• Have cloud (Azure, AWS, GCP) certification