Java Virtual Threads + Spring di Production: 6 Bulan

Enam bulan lalu, saya enable Virtual Threads di Spring Boot 3.4 production untuk klien enterprise (modular monolith hasil migration di Spring Boot modular monolith bank Indonesia). Workload mixed I/O + CPU, ~2.400 TPS peak. Tim sebelumnya struggle dengan thread pool tuning yang tidak pernah optimal.

Hari ini, share angka real Virtual Threads di production, 4 issue penting (pinning), dan kapan saya revert ke platform thread.

Konteks workload

• App: bank menengah Indonesia, modular monolith.
• Stack: Java 21 + Spring Boot 3.4 + Spring Modulith 1.3 + Postgres 17.
• Workload: 70% I/O-bound (DB call, HTTP downstream, Kafka), 30% CPU-bound (validation, transformation, signing).
• Throughput: 2.400 TPS peak, p99 latency target < 700ms.
• Hardware: 6 instance c6i.4xlarge (16 vCPU, 32GB RAM) di AWS Jakarta.

Sebelum Virtual Threads:

• Tomcat thread pool: 400 thread.
• Latency p99: 700ms (at SLA edge).
• Throughput ceiling: ~1.800 TPS sebelum thread pool saturation.
• Memory: 18GB RSS per instance (heavy thread overhead).

Enable Virtual Threads

Spring Boot 3.2+ punya native support. Cuma butuh property:

spring:
  threads:
    virtual:
      enabled: true

Yang ini enable:

• Tomcat akan pakai virtual thread untuk handle request.
• @Async task pakai virtual thread executor.
• Spring’s task scheduler pakai virtual thread.

Plus JVM flag:

-XX:+UnlockExperimentalVMOptions
-Djdk.tracePinnedThreads=short

tracePinnedThreads di staging only, production saya pakai JFR untuk overhead lebih rendah.

Cara kerja Virtual Threads di Spring

Setiap HTTP request di-handle oleh virtual thread (lightweight, ~few KB stack). Saat virtual thread melakukan blocking I/O (DB call, HTTP call), JVM unmount thread dari carrier (platform thread), free carrier untuk handle virtual thread lain. Setelah I/O complete, virtual thread mount kembali ke carrier untuk lanjut eksekusi.

Result: 1 carrier thread (platform) bisa handle ribuan virtual thread yang bergantian I/O wait.

Performance hasil

Test scenario: load test 4 jam sustained, traffic mirror dari production:

Metric	Platform Threads (400)	Virtual Threads	Delta
Throughput sustained	1.800 TPS	5.800 TPS	+222%
Latency p50	28ms	18ms	-36%
Latency p95	240ms	95ms	-60%
Latency p99	700ms	280ms	-60%
Memory RSS	18GB	7.6GB	-58%
CPU avg	62%	78%	+26%
Thread count active	380	12.000 (peak)	huge

CPU naik karena: utilization lebih tinggi. Sebelumnya thread pool starvation = idle CPU saat banyak thread blocked di I/O.

Memory turun karena: platform thread ~1MB stack default × 400 = ~400MB minimum dari thread metadata + stack. Virtual thread ~few KB stack on-demand.

Per instance, kami bisa downscale dari 6 instance c6i.4xlarge ke 4 instance c6i.2xlarge (sama throughput). Cost AWS turun 67%.

Yang break: Thread Pinning

Virtual thread tidak bisa unmount dari carrier saat:

1. Sedang dalam synchronized block.
2. Native method (JNI).
3. Beberapa class library legacy.

Saat pinning, virtual thread blokir carrier seperti platform thread biasa. Kalau semua carrier ter-pin → throughput drop ke baseline platform thread.

Pinning #1: BCrypt password hashing

Library BCryptPasswordEncoder Spring Security default pakai synchronized di hash function (untuk JCE crypto safety).

Detection: JFR show jdk.VirtualThreadPinned event dengan stack trace ke BCrypt.hashpw.

Impact: login endpoint p99 spike dari 95ms ke 380ms saat traffic burst.

Fix: gunakan Argon2 (Argon2PasswordEncoder) yang tidak synchronized. Migration smooth dengan Spring’s DelegatingPasswordEncoder:

@Bean
public PasswordEncoder passwordEncoder() {
    Map<String, PasswordEncoder> encoders = Map.of(
        "argon2", new Argon2PasswordEncoder(16, 32, 1, 65536, 3),
        "bcrypt", new BCryptPasswordEncoder()  // legacy verification
    );
    
    DelegatingPasswordEncoder encoder = new DelegatingPasswordEncoder("argon2", encoders);
    encoder.setDefaultPasswordEncoderForMatches(new BCryptPasswordEncoder());
    return encoder;
}

User existing tetap pakai bcrypt hash (verify only), user baru atau yang update password pakai argon2. Migrasi natural.

Pinning #2: Postgres JDBC driver versi lama

PostgreSQL JDBC driver < 42.7 punya synchronized di socket read.

Detection: JFR pinning event di PgConnection.executeWithFlags.

Impact: throughput plateau pada ~2.000 TPS saat seharusnya bisa 5.000+.

Fix: upgrade driver ke 42.7.4+ yang tidak synchronized:

<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.7.4</version>
</dependency>

Plus pakai connection pool yang virtual-thread friendly (HikariCP 5.1+ aware).

After fix: throughput sustainable 5.800 TPS.

Pinning #3: Apache Commons Logging

Beberapa library lama pakai org.apache.commons.logging.LogFactory yang punya synchronized init.

Detection: pinning saat first log call setelah app start (warm-up window).

Impact: spike latency 200ms-1s di 30 detik pertama post-startup.

Fix: pakai SLF4J binding langsung, avoid commons-logging:

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jcl-over-slf4j</artifactId>
</dependency>

Bridge JCL ke SLF4J, no synchronized.

Pinning #4: Custom legacy code

Internal library kami punya synchronized (CACHE_LOCK) block panjang (~50ms work). Itu memang kontra dengan virtual thread philosophy.

Fix: refactor ke ReentrantLock (Java 21 reentrant lock virtual-thread friendly):

// Sebelum
private static final Object CACHE_LOCK = new Object();

public void update(String key, Object value) {
    synchronized (CACHE_LOCK) {
        cache.put(key, value);
        // ... slow work
    }
}

// Setelah
private final ReentrantLock lock = new ReentrantLock();

public void update(String key, Object value) {
    lock.lock();
    try {
        cache.put(key, value);
        // ... slow work
    } finally {
        lock.unlock();
    }
}

ReentrantLock.lock() tidak pin carrier. Virtual thread bisa unmount saat lock contention.

ThreadLocal: still works, tapi consider scoped values

Virtual thread support ThreadLocal, tapi memory implication beda. Tiap virtual thread punya ThreadLocal entry tersendiri. Saat ada 12.000 virtual thread aktif, ThreadLocal × 12.000 instance.

Untuk request-scoped context (user ID, tenant ID, trace ID), saya migrate ke ScopedValue (Java 21 preview, 23 stable):

public static final ScopedValue<TenantContext> TENANT = ScopedValue.newInstance();

public PaymentResult process(Request req) {
    return ScopedValue.where(TENANT, resolveTenant(req))
        .call(() -> {
            // Inside this lambda, TENANT.get() available
            return paymentService.process(req);
        });
}

ScopedValue immutable, scope-bound, lebih cocok virtual thread paradigm. Memory savings ~120MB per instance setelah migrate ThreadLocal yang heavy.

Monitoring di production

Saya track tiga signal:

1. Pinned thread events

java -XX:StartFlightRecording=duration=1h,filename=/tmp/flight.jfr,settings=profile \
     -XX:FlightRecorderOptions=stackdepth=128 \
     ...

Lalu analyze:

jfr print --events jdk.VirtualThreadPinned /tmp/flight.jfr

Threshold: kalau pinning event > 100/menit, alert ke Slack.

2. Carrier thread saturation

Default carrier pool size = Runtime.availableProcessors() (16 di c6i.4xlarge). Saturation = semua carrier busy + virtual thread queue growing.

Metric via Micrometer + Prometheus:

@Bean
public MeterBinder threadMetrics() {
    return meterRegistry -> {
        ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
        Gauge.builder("jvm.threads.virtual.queued", () -> getVirtualThreadQueueSize())
            .register(meterRegistry);
    };
}

3. Latency per endpoint sebelum/sesudah enabled

Compare per-endpoint p99 latency. Endpoint dengan latency tidak improve atau worsen = kandidat investigation (mungkin CPU-bound atau pinned).

Kapan saya tetap pakai platform thread

1. CPU-bound workload: ML inference, image processing, encryption batch. Virtual thread tidak kasih benefit (tidak ada I/O wait untuk eksploit).

   Untuk endpoint compliance report generation yang CPU-heavy (PDF render dengan iText), saya buat dedicated @Async("cpuBoundExecutor") pakai ForkJoinPool platform threads.

2. Workload dengan synchronized block legacy yang tidak bisa refactor: kalau dependency vendor yang tidak bisa di-update.

3. Native library heavy: app yang call native code via JNI (image processing, ML model native).

Hasil 6 bulan

Metric	Sebelum	Sesudah
Instance count	6× c6i.4xlarge	4× c6i.2xlarge
Throughput sustained	1.800 TPS	5.800 TPS
P99 latency	700ms	280ms
Memory RSS per instance	18GB	7.6GB
Cost AWS monthly	$4.800	$1.580
Pinning incident	n/a	4 (di awal, all fixed)

Cost saving 67%, throughput naik 3.2x. Worth migrate.

Yang masih perlu hati-hati

• Database connection pool: HikariCP awal pakai size 50 per instance × 6 instance = 300 connection ke Postgres. Setelah virtual thread, throughput naik, connection demand naik. Saya tune ke 80 per instance × 4 instance = 320 connection. Postgres max_connections=400.

• Downstream service: kalau downstream pakai platform thread + thread pool ketat, throughput Anda yang naik bisa saturate downstream. Kami pernah saturate auth-service (yang belum migrate ke virtual thread) — fix dengan client-side rate limit.

• Debugging: thread dump dengan 12.000 virtual thread = wall of text. Pakai jcmd <pid> Thread.dump_to_file -format=json /tmp/dump.json untuk parse programmatically.

Verdict

Java 21 Virtual Threads di Spring Boot 3.4 production: worth migrate untuk workload I/O-bound. 3.2x throughput dan 67% cost saving signifikan untuk enterprise Indonesia.

Tapi: bukan plug-and-play. Pinning audit + ThreadLocal review + library upgrade adalah requisite. Untuk app yang baru, default to virtual threads. Untuk app legacy, plan 2-3 sprint untuk migrate dengan benar.

Bukan magic. Pinning issue selalu ada, JFR tooling adalah teman terbaik. Lihat juga Spring Boot modular monolith untuk arsitektur context yang underlying.