Senior Data Engineer

Summary

Join CircleCI as a Senior Data Engineer and play a pivotal role in developing and maintaining our data infrastructure. You will design and implement data architectures, build event-based logging systems, and establish data contracts to ensure data quality. This role requires collaboration with engineering and data teams to create robust, scalable data systems. As the initial data engineering hire, you will work independently while contributing to our data products, including data ingestion, AI/ML systems, and customer-facing APIs. You will leverage your expertise in Python, SQL, and modern data warehousing solutions to optimize data infrastructure costs and maintain performance. This is a unique opportunity to shape our data strategy and contribute to a data-driven company in the CI/CD space.

Requirements

• 5+ years of experience in data engineering roles
• Strong expertise in Python and SQL
• Proven experience with modern data warehousing solutions, particularly Snowflake
• Deep understanding of database systems, including PostgreSQL
• Hands-on experience with data integration tools like Fivetran
• Knowledge of CI orchestration tools like CircleCI or dbtCloud
• Proficiency with data pipeline orchestration platforms like Apache Airflow
• Experience with modern data transformation tools, specifically dbt
• Track record of implementing event-based logging systems
• Strong background in data modeling and architecture design
• Self-motivated with ability to work independently
• Excellent problem-solving and analytical skills
• Strong communication abilities to work effectively with both technical and non-technical stakeholders
• Experience in implementing and maintaining data quality standards
• Proven ability to optimize data infrastructure costs
• Experience with data contract design and implementation

Responsibilities

• Design and implement data architectures that support efficient data flow across the organization
• Build and maintain event-based logging systems to ensure comprehensive data capture
• Establish and enforce data contracts to maintain data quality and consistency
• Optimize data infrastructure costs while maintaining performance
• Create and maintain data pipelines using modern orchestration tools
• Partner with analytics engineers to ensure data availability and reliability
• Evaluate and implement new data tools and technologies as needed

Preferred Qualifications

• Experience being a founding member of a data engineering team
• Background in data security and compliance
• Experience with batch and microbatch data processing systems
• Knowledge of data observability tools and practices
• Experience with GPU infrastructure management and optimization
• Familiarity with ML model versioning and experiment tracking tools