Databricks Engineer, Healthcare

Summary

Join Blend360 as a Databricks Data Engineer to support client's healthcare analytics initiatives by designing, building, and optimizing data pipelines and workflows.

Requirements

• Strong experience in SQL, PySpark, CI/CD, CLI, Azure DevOps, Unity Catalog, various cluster settings, performance optimization, and end-to-end pipeline setup
• Experience with parsing and loading semi-structured data (JSON)
• Experience with parsing and loading unstructured data (clinical notes, DICOM images - radiology reports, MRI scans, etc.)
• Ability to set up streaming data in the Azure environment (Azure Event Hub)
• Prior healthcare experience
• Databricks certification

Responsibilities

• Design, build, and manage scalable data pipelines and ETL processes in Databricks
• Integrate data from various clinical, financial, and operational systems
• Develop and maintain the data lake and data warehouse environments
• Collaborate with data governance and compliance teams to ensure data privacy, security, and accuracy
• Work closely with data scientists, analysts, and business stakeholders to understand data needs
• Identify performance bottlenecks and optimize Databricks workflows for efficiency and cost-effectiveness
• Create and maintain clear, comprehensive documentation for data processes

Benefits

• Health insurance
• Vision benefits
• Dental benefits
• 401K plan
• Life Insurance
• Pretax Commuter Benefits