Databricks Architect

Summary

Join our team as a DataBricks Architect and work 100% remotely on long-term projects for our US-based client! We are seeking experienced professionals to contribute to AI/ML projects in the healthcare sector, focusing on analyzing textual and image data from medical devices. Your responsibilities will include data migration, processing, and optimization within the DataBricks cloud platform. The projects are primarily short-term, but with a high likelihood of extension, offering continuous work opportunities. The client prioritizes a collaborative environment and flexible work hours. Apply now to be part of a dynamic team tackling challenging data management problems!

Requirements

• Solid experience as a data engineer or in related roles (8+ years)
• Very good knowledge of the DataBricks and Apache Spark platform
• Very good knowledge of Python
• Experience in conducting cloud migrations
• Experience working in the AWS (Amazon s3) environment
• Experience in conducting projects related to AI/ML
• Interpersonal and teamwork skills
• Ability to take initiative and work independently
• English at a level that allows for fluent communication in a team

Responsibilities

• Ensure data security and storage
• Process and index DICOM data
• Validate data, create data processing pipelines, create and share cohorts
• Plan and execute database migrations
• Collaborate closely with the team (including data engineers, data scientists, clinical IT specialists, and support team)

Preferred Qualifications

• Experience working in the AWS Microsoft Azure environment (e.g., Data Factory, Synapse, Logic Apps, Data Lake)
• Experience in designing and optimizing data flows using DBT, SSIS, TimeXtender or similar solutions (ETL, ELT)
• Experience with any big data or NoSQL platforms (Redshift, Hadoop, EMR, Google Data, etc.)

Benefits

• Private medical care (Medicover)
• Multisport card for contractors