Principal Platform Data Engineer

Summary

Join 8th Light, a remote-first software consultancy, as a Principal Platform Data Engineer. Lead engagements focused on experimentation, pipelines, and reproducibility. Design, implement, and maintain robust data pipelines using Apache Airflow. Build BI dashboards and experimentation frameworks. Transform raw event data in AWS S3 into performant datasets in Snowflake and Athena. Collaborate across engineering, data, and product teams to deliver end-to-end pipelines and statistical insights. This role requires deep data infrastructure and experimentation experience, advanced Python and Java skills, and familiarity with AWS S3 and Snowflake.

Requirements

• Designed and maintained scalable systems with strong attention to performance, security, and reliability
• Advanced Python skills for statistical computation and building reusable libraries
• Proficiency in Java (Spring Boot) for backend service collaboration and API integration
• Experience building and monitoring Apache Airflow DAGs
• Deep familiarity with AWS S3, especially using Parquet, partitioning, and cost-effective storage patterns
• Shaped data and analytics capabilities, including advanced SQL optimization and Snowflake modeling
• Enabled insights through dashboards and Business Intelligence (BI) platforms like Tableau and Power BI
• Applied experimentation frameworks or data analysis to guide product decisions
• Led teams or projects and helped resolve ambiguity through clear decision-making
• Communicated effectively with both technical and non-technical stakeholders
• A curiosity-driven approach to evaluating and refining experimentation logic

Responsibilities

• Design, implement, and maintain robust data pipelines using Apache Airflow, managing both scheduled and on-demand compute workflows
• Building BI dashboards and experimentation frameworks that generate insights and business value
• Define and manage data contracts and schemas aligned with experimentation configurations and variant metadata
• Transform raw event data in AWS S3 into performant datasets in Snowflake and Athena, using efficient partitioning and modeling practices

Preferred Qualifications

• Support or lead efforts in advanced experimentation techniques such as Bayesian inference, regression modeling, or simulation frameworks
• Artificial Intelligence (AI) & Machine Learning (ML)

Benefits

• An L&D Program which includes: Learning budget, and in-person learning opportunities
• Coworking Access to support our remote-first team
• Wellness days
• 12 weeks of new parent leave available for eligible employees
• Semi-annual promotion panel