Remote Data Pipeline Engineer, ML Infrastructure Engineer

Summary

Join Trunk Tools in tackling the construction industry by designing and developing robust file processing pipeline infrastructure. As a data engineer, you will orchestrate data flow, ensure observability, and integrate with various sources. Collaborate with the machine learning team to enhance pipeline functionality.

Requirements

• BS/MS in Computer Science, Information Systems, or a related field
• 5+ years of experience in ETL development and data engineering
• Strong coding proficiency in Python and database systems (SQL, noSQL)
• Strong experience with pipeline orchestration tools (eg. Prefect), infrastructure-as-code (eg. Terraform), and observability and monitoring tools
• Understanding of serverless architectures (eg. AWS Lambda)
• Familiarity with ML workflows and requirements (to effectively collaborate with the ML team)
• Knowledge of data modeling and data warehouse concepts

Responsibilities

• Design, develop and maintain robust file processing pipeline infrastructure
• Orchestrate the flow of data through various stages of processing
• Ensure observability and monitoring of the pipeline’s health
• Integrate data from various sources including industry storage platforms, project management tools, and external APIs
• Implement data quality checks and error handling mechanisms to ensure data integrity
• Collaborate with the machine learning team to enhance pipeline functionality and efficiency

Benefits

• Competitive salary
• Stock option equity packages
• 3 Medical Plans to choose from including 100% covered option. Plus Dental and Vision Insurance!
• 401K
• Learning & Growth stipend
• Free lunch provided in NYC and Austin office - you’ll never go hungry with us!
• Unlimited PTO; We truly believe in work-life balance and that hard work should be balanced with time for rest and rejuvenation
• IRL / In-Person retreats throughout the year