Data/Ml Platform Engineer

Summary

Join Xebia, a global leader in digital solutions, as a Data Engineer. You will be responsible for developing and delivering high-quality features for a new ML Platform, refactoring data products, and completing various tasks to a high standard. This hands-on role requires expertise in big data, cloud technologies (AWS), and various ML frameworks. You will work closely with data scientists and contribute to establishing scalable, automated data processes. The position demands strong coding skills (Python, Spark), experience with cloud infrastructure, and excellent communication abilities. Immediate availability and EU work permit are required.

Requirements

• Be ready to start immediately
• Have 5+ years of hands-on experience in using Python, Spark
• Have experience in AWS Cloud usage and management
• Have experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow)
• Have experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch
• Have experience with orchestrators such as Airflow and Kubeflow
• Have familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes)
• Have a fundamental understanding of Parquet, Delta Lake and other data file formats
• Have proficiency on an IaC tool such as Terraform, CDK or CloudFormation
• Have strong written and verbal English communication skill and be proficient in communication with non-technical stakeholders
• Work from the European Union region and have a work permit

Responsibilities

• Develop and deliver high-quality features for our client's new ML Platform
• Refactor and translate our data products
• Complete various tasks to a high standard
• Be responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems
• Build and maintain architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies
• Evaluate and translate technical design to workable technical solutions/code and technical specifications at par with industry standards
• Drive creation of re-usable artifacts
• Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation
• Work closely with analysts/data scientists to understand impact to the downstream data models
• Write efficient and well-organized software to ship products in an iterative, continual release environment
• Contribute and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences
• Define data retention policies
• Monitor performance and advise any necessary infrastructure changes