Data/Ml Platform Engineer

Summary

Join Xebia, a global leader in digital solutions, as a Data Engineer. You will be responsible for developing and delivering high-quality features for a new ML Platform, refactoring data products, and completing various tasks to a high standard. This hands-on role requires expertise in big data and cloud technologies, including AWS, Databricks, and various ML frameworks. You will work closely with analysts and data scientists, contributing to the creation of reusable artifacts and scalable processes. The ideal candidate possesses 5+ years of experience with Python, Spark, and relevant cloud technologies, along with strong communication skills. This position is based in Moldova and requires immediate availability.

Requirements

• Be ready to start immediately
• Have 5+ years of hands-on experience in using Python, Spark
• Have experience in AWS Cloud usage and management
• Have experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow)
• Have experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch
• Have experience with orchestrators such as Airflow and Kubeflow
• Have familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes)
• Have a fundamental understanding of Parquet, Delta Lake and other data file formats
• Be proficient on an IaC tool such as Terraform, CDK or CloudFormation
• Have strong written and verbal English communication skill and be proficient in communication with non-technical stakeholders
• Currently reside in Moldova and hold the legal right to work in Moldova

Responsibilities

• Develop and deliver high-quality features for our client's new ML Platform
• Refactor and translate our data products
• Complete various tasks to a high standard
• Be responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems
• Build and maintain architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies
• Evaluate and translate technical design to workable technical solutions/code and technical specifications at par with industry standards
• Drive creation of re-usable artifacts
• Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation
• Work closely with analysts/data scientists to understand impact to the downstream data models
• Write efficient and well-organized software to ship products in an iterative, continual release environment
• Contribute and promote good software engineering practices across the team
• Communicate clearly and effectively to technical and non-technical audiences
• Define data retention policies
• Monitor performance and advise any necessary infrastructure changes