Staff Data Engineer, Lead Data Engineer

Summary

Join Machinify, a leading provider of AI-powered healthcare software, as a Staff/Lead Data Engineer. You will build and own critical data pipelines, collaborating with cross-functional teams to deliver a scalable framework. This role requires deep experience in data engineering, ETL orchestration, and distributed computing. You will map customer data, manage data quality, and work with various technologies like Apache Airflow, Spark, SQL, Python, and cloud platforms (AWS & GCP). The salary range is $220k-$270k, and the compensation package includes equity, excellent healthcare, flexible time off, and other benefits.

Requirements

• Deep experience as a hands-on Data Engineer building production data pipelines
• Experience managing the delivery of complex data
• Experience in ETL orchestration and workflow management tools with a strong preference for Apache Airflow
• Experience in Spark or other distributed computing frameworks
• SQL and Python
• Advanced SQL performance tuning
• Kubernetes and building Docker images
• AWS & GCP
• Experience working with APIs to collect or ingest data
• Streaming technologies like kafka , spark streaming etc
• ELK stack , Grafana etc

Responsibilities

• Independently understand all aspects of a business problem including those unrelated to their area of expertise, weigh pros and cons of different approaches and suggest ones likely to succeed
• Work with a cross-functional organization including engineering, delivering, subject-matter experts, product managers, as well as platform engineers to deliver a scalable framework
• Map the customer data into Machinify canonical form. Identify and ingest non canonical fields and generalize the process to a minimal level of customization
• Proactively design and adapt the canonical form to suit changing query patterns and needs
• Ultimately own data availability and quality for the Data Science organization
• Manage SLA for all pipelines in allocated areas of ownership

Preferred Qualifications

Experience in ETL orchestration and workflow management tools with a strong preference for Apache Airflow

Benefits

• Meaningful equity
• Excellent healthcare
• Flexible time off
• Other benefits and perks