DORA Metrics di Tim 3 Engineer SaaS Jakarta

Klien SaaS Jakarta saya operate (~600 paying tenant) punya tim 3 engineer fulltime. Owner minta saya assessment “tim kita produktif tidak?” 6 bulan lalu. Saya implement DORA metrics tracking sederhana. Hasil terukur, dan pattern yang berhasil scale untuk tim kecil tanpa overhead enterprise tooling.

Setup tracking

Tools yang saya pakai:

• GitHub API: source untuk commit, PR, deploy event
• Plausible Analytics: untuk track production deploy via custom event
• Sentry: untuk track incident + alert
• Grafana (self-hosted di Hetzner): visualize semua metric
• Bash + Bun script: nightly cron yang query API, write ke Postgres, push ke Grafana

Total tooling cost: ~$0 (semua existing infrastructure klien).

Implementation effort: 3 hari kerja saya, billed Rp 7,5jt one-time.

4 DORA metrics

1. Deploy Frequency

Frequency tim ship ke production. Pattern:

Cron query GitHub API setiap malam:

const deploys = await octokit.actions.listWorkflowRuns({
  owner, repo,
  workflow_id: 'deploy-prod.yml',
  status: 'success',
  created: `>${yesterday}`,
});

Insert ke deploys table dengan timestamp.

Baseline (Januari 2026): ~0,8 deploy/hari (sekitar 4 deploy/minggu).

6 bulan later (Juni 2026): ~3,2 deploy/hari (sekitar 16/minggu).

Improvement 4x. Source:

• Feature flag pattern (deploy backend + UI terpisah, flag-controlled rollout)
• Pipeline build time turun dari 12 menit ke 3,5 menit (parallel job + cache improvement)
• Trunk-based dev: PR small, merge cepat

2. Lead Time for Changes

Time dari first commit ke production deploy.

Saya track via GitHub: ambil first commit di branch, lalu deploy timestamp.

SELECT 
  pr.merged_at - first_commit.timestamp AS lead_time
FROM pull_requests pr
JOIN commits first_commit ON ...
WHERE pr.merged_at > NOW() - INTERVAL '30 days';

Baseline: median 6,2 hari (P95 14 hari)

Sekarang: median 2,0 hari (P95 4,5 hari)

Improvement 68%. Source:

• Smaller PR (rata-rata dari 480 LOC ke 180 LOC)
• Review SLA explicit (24 jam max, tim disiplin)
• Less time stuck di “waiting for QA” (we shifted left, dev test own work)

3. Change Failure Rate

Percentage of deploys yang cause incident or rollback.

Saya track via convention: PR/commit dengan label hotfix, rollback, atau incident-fix counted as failure-related. Plus Sentry incident yang triggered post-deploy within 4 jam window.

Baseline: 18% (sekitar 1 dari 5,5 deploy ada incident)

Sekarang: 9% (sekitar 1 dari 11 deploy)

Improvement 50%. Source:

• Better test coverage (dari 38% ke 64% di critical path)
• Staging environment dengan production-like data (related: postmortem deploy Jumat sore)
• Pre-deploy smoke check di CI

4. Mean Time to Recovery (MTTR)

Time dari incident detected ke service restored.

Track via Sentry alert timestamp + incident resolution timestamp (manual entry di Linear issue).

Baseline: median 78 menit (P95 4,2 jam)

Sekarang: median 38 menit (P95 1,8 jam)

Improvement 52%. Source:

• Better alerting (Sentry → Telegram dalam < 1 menit dari error spike)
• Runbook ditulis untuk top 5 common incident
• Rollback automation (1-command rollback via GitHub Actions)

Implementation detail tracking

Script bash + Bun nightly:

// scripts/dora-metrics.ts
import { Octokit } from '@octokit/rest';
import { Pool } from 'pg';

const octokit = new Octokit({ auth: process.env.GH_TOKEN });
const db = new Pool({ connectionString: process.env.DATABASE_URL });

async function trackDeploys() {
  const since = new Date(Date.now() - 24 * 60 * 60 * 1000);
  const { data: runs } = await octokit.actions.listWorkflowRuns({
    owner: 'klien-org',
    repo: 'app',
    workflow_id: 'deploy-prod.yml',
    status: 'success',
    created: `>${since.toISOString()}`,
  });
  
  for (const run of runs.workflow_runs) {
    await db.query(`
      INSERT INTO dora_deploys (deployed_at, commit_sha, workflow_run_id)
      VALUES ($1, $2, $3)
      ON CONFLICT (workflow_run_id) DO NOTHING
    `, [run.created_at, run.head_sha, run.id]);
  }
}

async function trackLeadTime() {
  // Query merged PR last 24h, compute lead time per PR
  // ...
}

await trackDeploys();
await trackLeadTime();
await trackIncidents();

Cron: 0 1 * * *.

Grafana dashboard: 4 panel (1 per metric), 30-day rolling average + 7-day window comparison.

Yang tim suka

1. Transparency: tim bisa lihat sendiri progress. Dorong improvement bottom-up.
2. Owner happy: data konkret untuk justify capacity hire atau training budget.
3. Onboarding effective: junior dev liat metric, paham target sustainable, bukan rush ship.

Yang bikin tim resist

1. Initial labeling discipline: tim sempat lupa label “rollback” atau “incident-fix”. Stats inaccurate selama 2 minggu pertama. Saya tambah PR template + bot reminder.

2. Metric gaming risk: deploy frequency naik karena tim split PR jadi terlalu kecil. Saya monitor “PR with no functional change” sebagai counter-metric.

3. Lead time skew dari long-lived feature branch: 1 PR yang ditahan 30 hari skew P95. Saya add filter untuk exclude outlier > 14 hari (counted as anomaly).

Cost & ROI

Setup cost (saya): Rp 7,5jt one-time. Maintenance: ~2 jam/bulan (saya audit metric, update script kalau API change).

Saving for klien:

• Less incident (50% reduction × ~Rp 4jt avg incident cost) = ~Rp 2jt/mo
• Faster delivery (lead time -68%) = approx feature velocity gain worth Rp 8-12jt/mo
• Lebih tinggi engineer retention (objective progress visibility = job satisfaction)

Klien very satisfied. Sekarang ini standard practice mereka.

Verdict

DORA metrics worth-it untuk tim engineer SMB 3-10 orang. Bukan untuk solo dev (overhead vs insight gain ratio rendah). Bukan untuk tim 50+ tanpa SRE function (butuh tooling proper, LinearB / Faros AI).

Bukan magic — improvement datang dari practice change (smaller PR, faster review, test coverage), bukan dari metric tracking itu sendiri. Metric expose problem, tim fix-nya.

Pattern simpler dari enterprise tooling, tetap actionable. Saya rekomendasi semua klien SaaS yang punya tim dev > 2 orang untuk implement variant pattern ini.