DataBricks Architect

Summary

Join our team as a DataBricks Architect for a client in the USA! This long-term, full-time, 100% remote position starts ASAP. You will work on projects involving data migration, collection, and optimization using DataBricks, primarily for healthcare AI/ML initiatives. Projects are typically short-term with high renewal probability, offering continuous work. Current projects focus on analyzing textual and medical image data (X-ray, MRI, etc.). The ideal candidate possesses strong Python skills, cloud experience, and expertise in DataBricks and Apache Spark.

Requirements

• Solid experience as a data engineer or in related roles (8+ years)
• Excellent knowledge of the DataBricks platform and Apache Spark
• Excellent knowledge of Python
• Experience in conducting cloud migrations
• Experience working in the AWS (Amazon s3) environment
• Experience in managing ML-related projects
• Interpersonal and teamwork skills
• Ability to take initiative and work independently
• English at a level that allows for fluent communication in a team

Responsibilities

• Ensure data security and storage
• Process and index DICOM data
• Validate data, create data processing pipelines, create and share cohorts
• Plan and execute database migrations
• Collaborate closely with the team (data engineers, data scientists, clinical IT specialists, support team)

Preferred Qualifications

• Experience working in the AWS Microsoft Azure environment (e.g., Data Factory, Synapse, Logic Apps, Data Lake)
• Experience in designing and optimizing data flows using DBT, SSIS, TimeXtender, or similar solutions (ETL, ELT)
• Experience with any big data or NoSQL platforms (Redshift, Hadoop, EMR, Google Data, etc.)

Benefits

• Private medical care (Medicover)
• Multisport card for contractors