DataBricks Engineer

Summary

Join our team as an Azure DataBricks Engineer and work on challenging projects for US clients! We are seeking experienced professionals with 8+ years of experience in data engineering or related roles for multiple short-term projects (with high potential for extension). These projects involve data migration, data collection, and optimization of DataBricks-based solutions. The client has a consistent need for specialists, offering new projects after completion of existing ones. Projects include migrating to DataBricks from SageMaker and internal platforms, and developing a platform for medical data analysis. Work is 100% remote, full-time, with flexible hours.

Requirements

• Solid experience as a data engineer or in related roles (8+ years)
• Excellent knowledge of DataBricks and Apache Spark
• Excellent knowledge of Python
• Experience in data migration
• Experience working in the Microsoft Azure environment (e.g., Data Factory, Synapse, Logic Apps, Data Lake)
• Experience working in the AWS environment
• Interpersonal and teamwork skills
• Ability to take initiative and work independently
• English at a level that allows for fluent communication in a team

Responsibilities

• Plan work and select appropriate tools
• Integrate databases in near real-time
• Create ETL processes
• Migrate databases, platforms, and ML models
• Optimize and automate platforms
• Collaborate closely with data engineers, data scientists, and architects

Preferred Qualifications

• Experience in designing and optimizing data flows using DBT, SSIS, TimeXtender, or similar solutions (ETL, ELT)
• Experience with any big data or NoSQL platforms (Redshift, Hadoop, EMR, Google Data, etc.)

Benefits

• Private medical care (Medicover)
• Multisport card for contractors