Databricks Architect

Summary

Join our team as a Databricks Architect for a US-based client working on an AI/ML healthcare project! This long-term, full-time, 100% remote position focuses on migrating, ingesting, and optimizing Databricks-based solutions. The project involves analyzing text and image data from medical devices and migrating it to a Databricks-powered cloud database. You will be responsible for the full data lifecycle, including compliance, auditing, and cohort creation. We are seeking engineers with 8+ years of experience in data engineering, strong Python skills, and extensive Databricks and Apache Spark experience. The work requires collaboration with a cross-functional team and offers flexibility in working hours.

Requirements

• 8+ years of experience in data engineering or related roles
• Strong hands-on experience with Databricks and Apache Spark
• Excellent knowledge of Python
• Experience with cloud migrations
• Hands-on experience with AWS (especially Amazon S3)
• Previous work on AI/ML projects
• Strong communication & teamwork skills
• A proactive mindset and independence
• English good enough for smooth daily communication

Responsibilities

• Ensuring secure data storage
• Processing and indexing DICOM data
• Validating data, building processing pipelines, creating and sharing cohorts
• Planning and executing database migrations
• Collaborating closely with a cross-functional team (data engineers, data scientists, clinical informaticians, support teams, etc.)

Preferred Qualifications

• Experience with Azure (Data Factory, Synapse, Logic Apps, Data Lake)
• Skills in designing/optimizing data pipelines using DBT, SSIS, TimeXtender or similar (ETL/ELT tools)
• Experience with big data or NoSQL platforms (e.g. Redshift, Hadoop, EMR, Google Data )

Benefits

• We work remote-first – travel is kept to a minimum
• We offer private medical care (Medicover) and Multisport card for contractors