Grafana Expert (Software Engineer)

Overview

We're looking for experienced Grafana power users to design expert-level evaluation tasks that test whether AI agents can use Grafana the way a real professional does. Your domain expertise is what makes these tasks authentic.

What You'll Do

• Design realistic, multi-step Grafana workflows - dashboards, alerting rules, data source configuration, panel setup, cross-module operations
• Perform each workflow yourself on a hosted Grafana instance

to produce a reference trajectory

• Write clear, specific task prompts with measurable outcomes that can be verified programmatically
• Implement programmatic graders that check whether each instruction was completed correctly
• Review AI agent attempts at your tasks, identify where and why they fail, and tag root causes
• Calibrate task difficulty so tasks are challenging but solvable - iterating on prompts and constraints based on model performance

Requirements

• 2+ years of daily, professional Grafana experience (SRE, Platform Engineering, Observability, or similar)
• Deep familiarity with PromQL, dashboard templating, alerting pipelines, and data source configuration (Prometheus, InfluxDB, etc.)
• Ability to articulate workflows clearly enough for programmatic verification
• Comfort writing basic grading scripts (Python; engineering support provided as needed)

Nice to Have

• Experience with Grafana API automation
• Kubernetes/infrastructure monitoring background
• Familiarity with AI evaluation or benchmarking

Time Commitment

• 10-15 hrs/week minimum during the project
• Fast turnaround expected - responsiveness matters