karawaci.kode

2026-07-06 · 8 min

Kubernetes Cost Cut 30%: Enterprise Jakarta

Enam bulan lalu, klien enterprise Jakarta (logistik B2B + e-commerce, ~14k internal user, ~480k order/hari) datang dengan tagihan AWS yang naik 38% YoY tanpa traffic growth yang proporsional. Cluster Kubernetes mereka — 142 node EKS, banyak service Java + Node — adalah cost driver utama. Bill Kubernetes: $48k/bulan dari total $112k AWS spend.

4 bulan optimasi, cost cluster jadi $33k/bulan (-31%). Zero SLA breach. Share metodologi, angka, dan trade-off.

Konteks cluster

  • Cluster: AWS EKS 1.31, multi-AZ (3 AZ di ap-southeast-3 Jakarta).
  • Node group: 4 group, mix instance type m6i/c6i/r6i, ukuran .xlarge sampai .4xlarge.
  • Workload: ~340 pod per cluster, 47 deployment, 6 statefulset (Kafka, Postgres, Redis cluster).
  • Tim: 8 engineer platform + 12 product engineer.
  • SLA produksi: 99.95% uptime per service kritikal, P95 latency < 250ms.

Baseline cost breakdown ($48k/bulan):

  • Compute (EC2): $36k
  • EBS volume: $4.8k
  • Network (NAT, ELB, data transfer): $4.2k
  • EKS control plane: $216
  • Misc (CloudWatch, Secrets Manager): $2.8k

Step 1: observability baseline (minggu 1-2)

Sebelum cut, harus tahu apa yang dipakai. Saya install:

  • Kubecost (open source, self-host) — cost allocation per namespace/deployment.
  • VPA recommender mode-only (tidak auto-apply) untuk recommendation right-size.
  • Karpenter (replace cluster-autoscaler) — better bin-packing.

Setelah 7 hari data collection:

NamespaceCost/bulanUtilization avg
order-service$8.2kCPU 18%, Mem 34%
logistics-engine$6.8kCPU 42%, Mem 61%
notification$3.4kCPU 8%, Mem 22%
reporting$5.1kCPU 24%, Mem 38%
platform (kafka, redis)$9.2kCPU 31%, Mem 47%
ml-pipeline$4.8kCPU 71%, Mem 89%
Others$10.5kmixed

Insight: 4 namespace dengan utilization < 25% adalah kandidat right-size pertama.

Step 2: right-sizing pod (minggu 3-6)

VPA recommender ngasih threshold P95 utilization per pod. Saya turunkan request:

Sebelum (order-service):

resources:
  requests:
    cpu: "2"
    memory: "4Gi"
  limits:
    cpu: "4"
    memory: "8Gi"

Setelah (berdasar P95 + 30% headroom):

resources:
  requests:
    cpu: "750m"
    memory: "1.5Gi"
  limits:
    cpu: "2"
    memory: "3Gi"

Total pod 142 → request CPU sum turun dari 284 vCPU ke 168 vCPU. Memory sum dari 540GB ke 285GB.

Karpenter consolidate node, jumlah node turun dari 142 ke 89.

Cost compute saving: ~$11k/bulan.

Trap yang harus diwaspadai

  1. JVM container: Java pod butuh -XX:MaxRAMPercentage=75 agar JVM respect cgroup limit. Saya temukan 12 pod Java yang tidak set ini — JVM ambil memory > limit, OOM kill saat right-size. Fix per pod.

  2. Burstable workload: notification service yang baseline CPU 8% sebenarnya spike ke 95% saat event broadcast (SMS blast). Request 1 vCPU dengan limit 4 vCPU. Saat 8 pod replica spike bersamaan, node bisa pressure CPU.

Saya pakai HPA dengan metric kafka_consumer_lag (custom metric via Prometheus Adapter):

metrics:
  - type: Pods
    pods:
      metric:
        name: kafka_consumer_lag_per_pod
      target:
        type: AverageValue
        averageValue: "1000"

Lebih responsif untuk burstable workload daripada CPU-based HPA.

Step 3: spot instance migration (minggu 7-10)

Bagian cost-cutting paling besar potensi tapi paling risky.

Strategi:

  • Workload stateless tolerant interrupt: notification, reporting batch, ml-pipeline (training). Pure spot.
  • Workload stateful atau latency-sensitive: order-service primary, logistics-engine, platform. On-demand + reserved instance.
  • Mix: 70% spot + 30% on-demand untuk pod yang bisa tolerate eviction tapi punya SLA.

Karpenter NodePool config:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-mixed
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["m6i", "m6a", "m7i", "c6i", "c6a", "c7i"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["ap-southeast-3a", "ap-southeast-3b", "ap-southeast-3c"]
      taints:
        - key: spot-tolerable
          value: "true"
          effect: NoSchedule
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 60s
    expireAfter: 168h  # rotate weekly

Penting: diversifikasi instance family + AZ untuk reduce eviction storm probability. AWS Spot Advisor estimate: 8% interruption rate, tapi practical observation kami ~3-4%.

Deployment yang opt-in spot pakai toleration:

tolerations:
  - key: spot-tolerable
    operator: Equal
    value: "true"
    effect: NoSchedule

PDB (PodDisruptionBudget) saya tighten:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: order-service-pdb
spec:
  minAvailable: 80%
  selector:
    matchLabels:
      app: order-service

minAvailable: 80% (bukan maxUnavailable: 1) lebih conservative untuk service dengan replica banyak.

Eviction handling

Spot eviction notice 2 menit advance. Saya pakai aws-node-termination-handler untuk graceful drain:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: aws-node-termination-handler
spec:
  template:
    spec:
      containers:
        - name: aws-node-termination-handler
          image: public.ecr.aws/aws-ec2/aws-node-termination-handler:v1.22
          env:
            - name: ENABLE_SPOT_INTERRUPTION_DRAINING
              value: "true"
            - name: TAINT_NODE
              value: "true"
            - name: NODE_TERMINATION_GRACE_PERIOD
              value: "120"

Cost compute saving dari spot: ~$8.5k/bulan.

Step 4: EBS gp3 + storage class tuning (minggu 11)

EBS volume mostly gp2. Migrate ke gp3 dengan baseline IOPS yang sesuai workload:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3-balanced
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"
  encrypted: "true"
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

gp3 hanya $0.08/GB vs gp2 $0.10/GB, plus IOPS dipisah dari ukuran disk. Untuk Kafka broker yang butuh IOPS tinggi (~6000 IOPS), saya provision IOPS explicit tanpa over-provision disk size.

EBS saving: ~$1.2k/bulan.

Step 5: network egress optimization (minggu 12-14)

NAT Gateway $0.045/GB processed + per-AZ × 3. Bill NAT $3.8k/bulan.

Saya identifikasi traffic egress:

  • Image pull ECR via NAT → ganti ke VPC endpoint untuk ECR (saving $1.1k/bulan)
  • S3 cross-region traffic → ganti ke S3 Gateway VPC endpoint ($0/GB, saving $640/bulan)
  • DynamoDB call → DynamoDB Gateway VPC endpoint (saving $280/bulan)

Total network saving: ~$2k/bulan.

Step 6: reserved instance + savings plan (minggu 15-16)

Setelah workload stable, baseline on-demand compute usage konsisten ~95 vCPU continuous.

Buy Compute Savings Plan 1-year, no upfront, $1.8/hour:

  • Cover ~62 vCPU baseline.
  • Saving rate ~32% vs on-demand.
  • Cost saving: ~$3k/bulan.

Sisa burst pakai on-demand atau spot.

Hasil 4 bulan

ItemSebelumSesudahDelta
Compute (EC2)$36k$20k-44%
EBS$4.8k$3.6k-25%
Network$4.2k$2.2k-48%
EKS + misc$3.0k$3.0k0%
Total/bulan$48k$33k-31%
Node count14271-50%
Pod count3403400
P95 latency (order-service)180ms195ms+8%
SLA breach00-

P95 latency naik 8% (tolerable, masih dalam SLA < 250ms) karena CPU headroom lebih ketat. Trade-off worth.

Yang break

1. Spot eviction storm hari ke-23

Hari Selasa siang, AWS spot price m6i.2xlarge ap-southeast-3 spike → 28 node spot di-evict dalam window 4 menit. Pod scheduling pressure, ~85 pod pending selama 6 menit.

Customer-visible: 4 menit elevated latency p99 (700ms vs baseline 240ms), zero error (PDB hold).

Mitigasi setelah ini:

  • Diversifikasi 3 instance family (sebelumnya 1)
  • Tambahkan on-demand fallback NodePool prioritas tinggi
  • Karpenter disruption.budgets set limit eviction concurrent
disruption:
  budgets:
    - nodes: "10%"
      reasons: ["Drifted", "Underutilized"]

2. JVM OOM setelah right-size

Service Java legacy (Spring Boot 2.7) di-right-size dari 4GB ke 2GB request. JVM -Xmx2g set, tapi overhead non-heap (metaspace, threads) ~800MB. Pod hit cgroup limit, OOM kill.

Fix:

env:
  - name: JAVA_TOOL_OPTIONS
    value: "-XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=50"

Plus tambah request memory dengan headroom realistic (2.5GB request, 3GB limit). Saving lebih kecil tapi stabil.

3. PDB blocking cluster upgrade

Saat EKS auto-upgrade ke 1.32, node drain stuck karena PDB conservative (minAvailable: 80%) di service dengan cuma 3 replica. 80% dari 3 = 2.4, ceiling 3, jadi tidak boleh evict satupun.

Fix: PDB review per deployment, pakai maxUnavailable: 1 untuk replica < 5, minAvailable: 80% untuk replica >= 5.

4. Karpenter consolidation aggressive

Karpenter consolidate node tiap 60 detik. Saat ada batch job pendek (~3 menit), Karpenter spawn node baru, lalu consolidate balik. Churn rate node tinggi → biaya per-second billing 1 menit minimum × banyak instance.

Fix: consolidateAfter: 300s (5 menit), plus separate NodePool untuk batch workload dengan TTL lebih panjang.

Kapan optimasi berhenti

Setelah 4 bulan, saving curve mulai diminishing return. Optimasi lanjutan butuh:

  • Rewrite service ke language lebih efficient (Go dari Java) — investment > 6 bulan engineering
  • Migrate ke serverless (Lambda) untuk burst workload — partial migration possible tapi observability rumit
  • Multi-cluster federation untuk workload isolation — complexity tinggi

Kami stop di 31% saving. Cost monitoring jalan terus, alert kalau drift > 10% dari baseline.

Verdict

Kubernetes cost optimization 30% realistis dalam 3-6 bulan tanpa rewrite. Yang dibutuhkan: observability (Kubecost), disiplin right-sizing per workload, dan kemauan handle spot eviction operasional.

Tidak semua cluster cocok untuk spot — workload latency-critical < 50ms p99 atau dengan PDB sangat ketat tetap on-demand. Tapi untuk mixed workload enterprise: spot 60-70% mix biasanya feasible.

Bukan magic. Saya butuh on-call 4 bulan rotasi cycle untuk monitor eviction storm. Saving $15k/bulan, biaya engineering ~$30k untuk project. ROI 2 bulan, payback indefinite. Konteks observability terkait lihat juga Prometheus Grafana SaaS setup.

Ditulis oleh Reza Pradipta