Senior Backend Engineer, Observability: Ingest

Summary

Join Grafana Labs as a Senior Backend Engineer, Observability: Ingest and contribute to the development and maintenance of key infrastructure for monitoring hundreds of thousands of observability collector instances worldwide. Collaborate with a team to deliver new features, lead projects from concept to implementation, and actively contribute to open-source projects like Grafana Alloy and the OpenTelemetry Collector. Design, build, operate, and maintain essential systems, ensuring reliability and performance. Take an active role in shaping the company roadmap and mentoring team members. This remote position requires strong communication skills and the ability to work independently. The ideal candidate is a motivated self-starter with a passion for creating intuitive products and a pragmatic approach to problem-solving.

Requirements

• Solid experience with at least one programming language. We use Go, but if you have familiarity with Python, C, C++, Rust or similar then that translates well
• Experience with delivering projects, from gathering requirements, brain-storming ideas all the way to shipping a product to the customer’s hands in a self-driven way
• Experience with developing software that runs in the Cloud or some experience with systems engineering
• Experience with being on-call and performing operations/SRE tasks or with the concept of infrastructure as code
• You are a motivated self-starter with a bias toward action
• You have a passion for creating intuitive products that fit customers’ needs
• While the vast majority of your work will be focused on the backend of 1 product, you don’t shy away from making bugfixes and other small changes to other projects (incl. frontend)
• Pragmatism: You are able to take on complex challenges and break them down to achieve short feedback loops: to analyze, design, and build modular solutions, deliver MVPs, gather data and feedback and then progress iteratively
• Collaboration and communication: The smallest unit we have is a team. You’ll be working with your teammates in a fully remote setup. Good communication skills are a must

Responsibilities

• Collaborate with your team to deliver new features, analyze outcomes, and make improvements
• Lead projects from concept to implementation, including ongoing customer support
• Become an active contributor of open source projects like Grafana Alloy and the OpenTelemetry Collector
• Design, build, operate, and maintain essential systems, ensuring reliability, performance, and availability
• Take an active role in influencing our roadmap and your own career objectives
• Participate in on-call rotations and take responsibility for the services you oversee
• Support and mentor team members, engage in design conversations, and work closely with colleagues
• Expand your skill set by deepening your knowledge of our cloud products, understanding our customers, and learning about our codebase

Preferred Qualifications

• Experience working with Kubernetes
• Been a user of Grafana and Prometheus in operational roles (including on-call for your team at a previous employer or just using these tools on hobby/homelab projects)
• Exposure to microservices architecture and distributed systems

Benefits

• Equity
• Bonus (if applicable)