build-grafana-dashboards

Files (1)

• SKILL.md

SKILL.md (first lines)

# Build Grafana Dashboards

Design and deploy Grafana dashboards with best practices for maintainability, reusability, and version control.

## When to Use

- Creating visual representations of Prometheus, Loki, or other data source metrics
- Building operational dashboards for SRE teams and incident responders
- Establishing executive-level reporting dashboards for SLO compliance
- Migrating dashboards from manual creation to version-controlled provisioning
- Standardizing dashboard layouts across teams with template variables
- Creating drill-down experiences from high-level overviews to detailed metrics

## Inputs

- **Required**: Data source configuration (Prometheus, Loki, Tempo, etc.)
- **Required**: Metrics or logs to visualize with their query patterns
- **Optional**: Template variables for multi-service or multi-environment views
- **Optional**: Existing dashboard JSON for migration or modification
- **Optional**: Annotation queries for event correlation (deployments, incidents)

## Procedure

> See [Extended Examples](references/EXAMPLES.md) for complete configuration files and templates.


### Step 1: Design Dashboard Structure

Plan dashboard layout and organization before building panels.

Create a dashboard specification document:

```markdown
# Service Overview Dashboard

## Purpose
Real-time operational view for on-call engineers monitoring the API service.

## Rows
1. High-Level Metrics (collapsed by default)
   - Request rate, error rate, latency (RED metrics)
   - Service uptime, instance count
2. Detailed Metrics (expanded by default)
   - Per-endpoint latency breakdown
   - Error rate by status code
   - Database connection pool status
3. Resource Utilization
   - CPU, memory, disk usage per instance
   - Network I/O rates
4. Logs (collapsed by default)
   - Recent errors from Loki
   - Alert firing history

## Variables
- `environment`: production, staging, development
- `instance`: all instances or specific instance selection
- `interval`: aggregation window (5m, 15m, 1h)

## Annotations
- Deployment events from CI/CD system
- Alert firing/resolving events
```

Key design principles:
- **Most important metrics first**: Critical metrics at the top, details below
- **Consistent time ranges**: Synchronize time across all panels
- **Drill-down paths**: Link from high-level to detailed dashboards
- **Responsive layout**: Use rows and panel widths that work on various screens

**Got:** Clear dashboard structure documented, stakeholders aligned on metrics and layout priorities.

**If fail:**
- Conduct dashboard design review with end users (SREs, developers)
- Benchmark against industry standards (USE method, RED method, Four Golden Signals)
- Review existing dashboards in team for consistency patterns

### Step 2: Create Dashboard with Template Variables

Build the dashboard foundation with reusable variables for filtering.

Create dashboard JSON structure (or use UI, then export):

```json
{
  "dashboard": {
    "title": "API Service Overview",
    "uid": "api-service-overview",
    "version": 1,
    "timezone": "browser",
    "editable": true,
    "graphTooltip": 1,
    "time": {
      "from": "now-6h",
      "to": "now"
    },
    "refresh": "30s",
    "templating": {
      "list": [
        {
          "name": "environment",
          "type": "query",
          "datasource": "Prometheus",
          "query": "label_values(up{job=\"api-service\"}, environment)",
          "multi": false,
          "includeAll": false,
          "refresh": 1,
          "sort": 1,
          "current": {
            "selected": false,
            "text": "production",
            "value": "production"
          }
        },
        {
          "name": "instance",
          "type": "query",
          "datasource": "Prometheus",
          "query": "label_values(up{job=\"api-service\",environment=\"$environment\"}, instance)",
          "multi": true,

From pjt222/agent-almanac, MIT.

More from pjt222/agent-almanac