Summary
Join our team as a Data Engineer and contribute to the migration of ML components from AWS SageMaker to Databricks! We are seeking experienced Data Engineers with a strong background in Machine Learning to work on a long-term, fully remote project for a US-based client. The project involves migrating and optimizing cloud-based models, requiring collaboration with a data/platform team. The work will be primarily during standard business hours, with flexibility in scheduling. This is a long-term opportunity with potential for continued work on similar projects within the same environment. The client values open communication and a streamlined recruitment process.
Requirements
- 7 years of experience as a Data Engineer
- Excellent knowledge of the Databricks platform and the Apache Spark framework (especially PySpark)
- Experience in migrating ML solutions (e.g., from SageMaker to Databricks) and transferring pipelines to a cloud environment
- Practical experience with AWS services, especially S3, EC2, SageMaker
- Participation in projects based on machine learning or AI, also in production environments
- Excellent knowledge of Python
- Experience with cloud platforms such as AWS or Azure
- English at a level that allows for fluent communication in a team
Responsibilities
- Migrate ML components from AWS SageMaker (training jobs, endpoints) to Databricks
- Deploy models to production, monitor metrics, and perform retraining
- Collaborate with the data/platform team on integrating models with existing data architecture
- Prepare an implementation plan for live deployment
- Automate and optimize ETL processes
Preferred Qualifications
- Knowledge of Microsoft Azure cloud services (e.g., Data Factory, Synapse, Logic Apps)
- Experience with MLflow, Feature Store, Delta Lake, CI/CD models, drift monitoring
- Experience working with big data or NoSQL databases (e.g., Redshift, EMR, Hadoop, Google BigQuery)
- General understanding of data pipeline management tools (DBT, Airflow, SSIS, etc.) - as a complement to collaboration with the Data Engineering team
Benefits
- Private medical care (Medicover)
- Multisport card for contractors