Staff Software Engineer, Workflow Orchestration

Summary

Join Airbnb's Workflow Orchestration team as a Data Engineer and contribute to the development and maintenance of data workflow orchestration solutions. You will collaborate with cross-functional teams to design and build frameworks supporting data development workflows, automation tools, and data engineering tooling. This role requires extensive experience in software engineering, data infrastructure, and workflow orchestration solutions. You will leverage your expertise in data modeling, database design, and various SQL dialects to optimize processes and introduce new technologies. The position offers a competitive salary, bonus, equity, benefits, and Employee Travel Credits and is US-Remote Eligible with occasional office work.

Requirements

• 10+ years of experience in software engineering, with a focus on data infrastructure or supporting data engineers and/or data scientists
• Strong programming skills in one or more of the following languages: Java, Python, or Scala
• Experience building data applications that create and implement higher-level abstractions on top of the lower-level data infrastructure to simplify complex data operations, improve scalability, and enhance overall system performance
• Experience working with data storage and distributed processing technologies (i.e. Hive, Spark, Trino, Flink or other SQL databases, etc …)
• Experience with data modeling, database design and various SQL dialects
• Experience with containerization technologies such as Docker and Kubernetes
• Extensive background in workflow orchestration solutions such as Apache Airflow, Luigi, Azkaban, Oozie, Prefect or Kubeflow
• Excellent communication skills and the ability to collaborate with cross-functional teams to build empathetic tools and platforms
• Excellent analytical and problem solving skills

Responsibilities

• Work with data infra engineers, data engineers, data scientists and product managers to design and build frameworks to support data development workflows
• Develop and maintain automation tools to streamline deployment and management of our platform
• Maintain and evolve data engineering tooling to ensure high availability, reliability, usability and performance
• Collaborate with data engineering and analytics teams to understand their needs and requirements to create a paved path for interacting with the data warehouse
• Design & implement frameworks (applications) for processing data at scale such as a metrics platform, SQL generation framework or a data quality framework
• Utilize experience in Data Engineering to optimize processes and bring forth new ideas and technologies that enhance our data capabilities

Preferred Qualifications

BS/MS/PhD in Computer Science, a related field, or equivalent work experience

Benefits

• Bonus
• Equity
• Benefits
• Employee Travel Credits