Remote Senior Data Engineer I

Summary

Join Grafana Labs as a Senior Data Engineer to maintain and develop data frameworks and systems. You will build and maintain data pipelines, implement data quality checks, and develop machine learning pipelines. This role involves collaboration with various teams across the company and utilizes technologies like GCP, BigQuery, dbt, and Prefect. You will contribute to improving data accuracy, reliability, and efficiency for sales, revenue, and usage metrics forecasting. The position requires strong software development skills, proficiency in SQL, and experience with data ingestion pipelines. Compensation includes a competitive salary and benefits.

Requirements

• Software development skills (some combination of Python, Java, Scala, Go)
• High proficiency in SQL
• Experience building and maintaining data ingestion pipelines using a workflow orchestration system (e.g. Prefect, Dagster, Airflow)
• Working knowledge of MLOps best practices
• Working knowledge of dbt or similar data transformation tools
• Highly motivated self-starter that is keen to make an impact and is unafraid of tackling large, complicated problems
• Excellent communication skills, able to explain technical topics to non-technical audiences, and maintain many of the essential cross-team and cross-functional relationships necessary for the team’s success

Responsibilities

• Build and maintain production quality data pipelines between operational systems and BigQuery (ingress and egress)
• Implement data quality and freshness checks and monitoring processes to ensure data accuracy and consistency
• Build and maintain machine learning pipelines to automate model validation and deployment
• Create and maintain comprehensive documentation for data engineering processes, systems, and workflows
• Maintain observability and monitoring of our internal data pipelines
• Troubleshoot and resolve data pipeline issues to ensure downstream data availability
• Contribute to our dbt systems by making sure the source and staging layers align with our standards, are efficient, cost-effective, and highly available

Preferred Qualifications

• Experience working with Prefect, BigQuery, and GCP services
• Knowledge about observability
• Previous experience with Grafana visualization, or a desire to invest the time to learn

Benefits

• Equity
• Bonus (if applicable)