Benchmarks¶
Evidence, not a performance promise. These measurements test whether the prototype's behaviour is plausible and where it loses. They are not a throughput guarantee, a capacity plan or a comparison of language ergonomics.
The roadmap asks for a number recorded at every milestone, so that the performance discussion is about measurements instead of guesses. The important rows are not only the fast ones: the tables below name the operations where BEAM is substantially ahead.
This is the first baseline, recorded at the end of 0.5. It is a starting point to detect regressions against, not a claim of speed.
What the harness does¶
BenchTest runs a warmup, then a measured batch, five times per workload, and
reports the median ns/op. Results are printed, never asserted: a shared machine
can not carry a performance budget, and a benchmark that fails the build turns
into a test that gets disabled.
Two execution modes are measured deliberately:
- Inline workloads bind a
Processto the benchmark thread, soModule.applyruns the opcodes on that thread. Process spawn and scheduler hand-off are excluded, so the number is the interpreter. - Cross-boundary workloads call an exported function from a Java thread
through
Erlang.apply. The process is spawned on a scheduler and the caller waits, so this is the cost an embedding application pays.
Every row also reports the change in stack depth per call. It is 0 everywhere
below, which is the check that a repeated call is not leaking stack frames.
Running it¶
The suite is skipped unless -Djerlang.bench=true is set, because it takes
about 15 seconds and measures the machine it runs on rather than the build.
The distribution codec and loopback transport have their own opt-in rows:
Distribution transport¶
Measured on 2026-11-02 with an 80-byte frame (control term plus a 32-byte binary message):
| Workload | ns/op | ops/sec | What it measures |
|---|---|---|---|
| Distribution encode + decode | 1,589.7 | 629,031 | 112-tagged frame, no socket |
| Distribution loopback round trip | 44,975.6 | 22,234 | 112-tagged frame, socket and peer thread |
The codec row isolates encoding and decoding. The loopback row adds the kernel TCP stack and a peer thread, but still excludes a second JVM and the scheduler hand-offs of a full node-to-node run. It is a transport measurement, not a claim about distribution against BEAM.
The baseline¶
Measured on 2026-10-29, at the end of milestone 0.5, on two consecutive runs. Run-to-run variation is about 10%, and larger for the closure workload.
Environment: 12th Gen Intel Core i5-12500H, 16 logical processors, 16 GiB, Windows 11 Home, Temurin OpenJDK 21.0.12.1, default JVM flags, no isolation from other processes.
| Workload | Run 1 (ns/op) | Run 2 (ns/op) | What it measures |
|---|---|---|---|
Module load (constructs_demo.beam) |
321,961 | 315,389 | Chunk parsing, instruction decode, export |
| Function call | 705 | 661 | One interpreted call, one comparison opcode |
| Guard dispatch | 395 | 377 | Call, guard test, clause selection |
List head (get_hd) |
377 | 332 | Pattern match on a list |
| Binary construct | 841 | 833 | bs_create_bin with an integer and a binary segment |
| Record update | 398 | 410 | update_record |
| Float binary segment | 1,170 | 1,163 | bs_create_bin with a float field |
| Big integer | 271 | 264 | 256-bit integer construction |
| Closures and higher order | 3,382 | 3,855 | Fun creation, capture, list comprehension, higher-order calls |
lists:reverse/1, 100 elements |
680 | 621 | The built-in (Java) module path |
| Java to Erlang call | 107,991,760 | 106,968,540 | Spawn on a scheduler and wait for the result |
| Java to Erlang round trip | 525,043,420 | 509,499,660 | Spawn, two messages, two receives, exit |
Sub-microsecond numbers carry a caveat: at this scale the measurement includes
two System.nanoTime calls amortised over the batch and the list construction
done once outside it, so treat them as order of magnitude.
After removing the scheduler poll¶
The first baseline found a 108 ms floor under every call from Java. It was not Erlang: an idle scheduler slept for 100 ms, and the caller polled the spawned process every 10 ms on top of that. Both were replaced:
Schedulerparks on a condition andadd()signals it, so a process on the run queue is picked up immediately instead of at the next poll.Module.applywaits on a latch the scheduler counts down when the process is done, instead of sleeping and re-checking the state.
Same machine, same harness, after the change:
| Workload | Before | After | Change |
|---|---|---|---|
| Java to Erlang call | 108.0 ms | 0.09-0.19 ms | ~580-1,200x faster |
| Java to Erlang round trip | 525.0 ms | 0.46-0.74 ms | ~700-1,100x faster |
The cross-boundary rows are dominated by OS scheduling and vary more between
runs than the others, hence the range. The upper ends (0.19 ms and 0.74 ms)
come from four repeat runs of BenchTest on 2026-10-29 with other processes
loading the machine; the lower ends are from a quieter run of the same harness.
A single run of either row is only good to about 30%.
Interpreted workloads are unchanged within noise (call 520-680 ns, guard
315-460 ns, list head 270-340 ns, binary construct 630-820 ns, record update
330-410 ns, float segment 970-1,120 ns, big integer 230-270 ns, closures
2.6-3.9 us, lists:reverse/1 540-680 ns).
What is left in the 0.1-0.2 ms is OS scheduling: the process runs on a scheduler thread and the caller waits for it, so two thread hand-offs are paid. That is inherent to the current design, which runs every process on a scheduler, and not a polling artefact. Further work would be a different architecture (running a call on the calling thread), not a smaller sleep.
CrossBoundaryCallTest fails if a cross-boundary call takes more than 25 ms,
which leaves two orders of magnitude of headroom over the measured 0.1 ms while
still catching a reintroduced poll. It was checked against the old behaviour
(75 ms median, failed).
Process scale and a ring (2026-11-05)¶
The 1.0 tables above measure small operations. The
concurrency example asks a different question: can the
runtime carry many live processes and a message token through a ring? The same
.erl module is compiled by JErlang and by erlc.
Environment: Windows 11, 12th Gen Intel Core i5-12500H, 16 GiB, Temurin OpenJDK
21.0.12.1, Erlang/OTP 29.1.2 (ERTS 17.1.1), no isolation; JErlang at -Xmx4g.
Each command was run once, so these are order-of-magnitude results.
| Workload | JErlang | BEAM |
|---|---|---|
| Spawn and join 5,000 processes | 20,872.7 ms | 7 ms |
| Spawn and join 100,000 processes | did not finish within 5 min | 393 ms |
| Spawn and join 1,000,000 processes | did not finish within 5 min | 59,000 ms |
| Ring, 1,000 processes × 1,000 messages | 348.0 ms | 2 ms |
| Ring, 1,000 processes × 10,000 messages | not run to completion | 7 ms |
This is the clearest loss in the table. Process creation is already about 2,980x BEAM's cost at 5,000 processes, and the prototype does not reach the million-process case. The 5,000-process run passes as a correctness test; the larger runs are reported as limits, not extrapolated. The BEAM million-process number is also superlinear on Windows (300,000 took 4.6 s; 600,000 took 22.5 s), so it is evidence that BEAM completes the workload, not a clean scaling curve. JErlang's current process representation and scheduler are the next performance target, and this result names the gap.
What the evidence says¶
- Interpreted operations cost hundreds of nanoseconds. A call is about 520 ns, a
guard about 320 ns, binary construction about 680 ns. The built-in
lists:reverse/1over 100 elements is about 540 ns, the same order as a single interpreted call: the Java implementations are not the bottleneck. - Loading a module is about 260 us: chunk parsing and instruction decode, paid once per module.
- A call across the Java/Erlang boundary costs about 0.1-0.2 ms, of which most is the two thread hand-offs. That is usable for control-plane work and still too slow for a per-item inner loop, which is what the roadmap's embedding target assumes.
Against BEAM¶
1.0.0 measurement¶
BeamBenchTest closes the gap the 0.5 baseline left: it compiles
src/test/erl/bench/beam_bench.erl with erlc, runs main/1 on a real OTP 29
node to get the BEAM column, then loads the same .beam file into this runtime
and times the same exported functions. Both columns are the same Erlang source,
the same machine and the same batch sizes, so the ratio is the only claim being
made and it is a ratio of two measurements taken minutes apart.
Measured on 2026-11-01, JErlang 1.0.0 against OTP 29 (ERTS 17.1.1), on the same machine as the 0.5 baseline:
| Workload | JErlang ns/op | BEAM ns/op | Ratio |
|---|---|---|---|
| Function call | 211.4 | 6.7 | 31.6x |
| Guard dispatch | 187.4 | 6.7 | 28.0x |
List head (get_hd) |
184.0 | 6.7 | 27.5x |
| Binary construct | 178.7 | 6.7 | 26.7x |
| Record update | 179.9 | 6.7 | 26.9x |
| Float binary segment | 177.9 | 6.1 | 29.2x |
| Big integer | 172.2 | 6.1 | 28.2x |
| Closures and higher order | 16,355.1 | 76.8 | 213.0x |
lists:reverse/1, 100 elements |
24,394.9 | 640.0 | 38.1x |
| Message round trip (spawn, 2 messages) | 439.3 | 2,252.5 | 0.20x |
| ETS insert + lookup | 4,118.5 | 819.0 | 5.0x |
| Startup, process wall clock | 242.1 ms | 167.1 ms | 1.4x |
3.0.0 release-candidate measurement¶
The same harness, same source and same machine were run again on 2026-11-05 with JDK 21.0.12.1, 16 processors, Windows 11 and OTP 29.1.2. The message throughput gate was enabled and passed:
| Workload | JErlang ns/op | BEAM ns/op | Ratio |
|---|---|---|---|
| Function call | 256.8 | 6.7 | 38.33x |
| Guard dispatch | 219.2 | 7.2 | 30.44x |
List head (get_hd) |
224.6 | 6.7 | 33.52x |
| Binary construct | 216.5 | 6.7 | 32.31x |
| Record update | 213.0 | 7.7 | 27.67x |
| Float binary segment | 201.0 | 7.2 | 27.92x |
| Big integer | 185.1 | 6.1 | 30.34x |
| Closures and higher order | 18,696.2 | 76.8 | 243.44x |
lists:reverse/1, 100 elements |
27,024.4 | 681.0 | 39.68x |
| Message round trip (spawn, 2 messages) | 572.4 | 2,508.5 | 0.23x |
| ETS insert + lookup | 7,846.6 | 1,024.0 | 7.66x |
| Startup, process wall clock | 485.0 ms | 205.8 ms | 2.36x |
Read the ratio, not the absolute numbers: a laptop under load moves both columns. Two rows deserve a note.
- Message passing is the one row where this runtime is ahead, at about one fifth of BEAM's cost for a spawn plus a two-message round trip. It is measured the way the embedding caller pays for it, with the spawned process on a scheduler, so the number includes the hand-off. It is also the row where the comparison is most sensitive to the workload: BEAM's figure includes its process exit path, which this runtime's lighter process shutdown skips.
- Closures are the outlier at 213x. Fun creation, capture and higher-order calls cost two orders of magnitude more than BEAM. That is the interpreter's next target and it is why the budgets below are per-category instead of one number.
Budgets¶
The roadmap asks for budgets set from the 1.0 baseline. Only the message throughput budget is enforced in CI, because it is the one the roadmap names and the one this runtime is comfortably inside:
- Message throughput: no more than 5x BEAM. Measured 0.20x in 1.0 and 0.23x
in the 3.0.0 candidate.
BeamBenchTestfails the build when-Djerlang.bench.gate=trueand the ratio exceeds-Djerlang.bench.max.ratio(default 5). - Interpreted single operations are recorded, not gated: CI runners vary by more
than the gap between two consecutive runs on one machine, so a gate there
would be a flaky test rather than a budget. The recorded targets are 35x for
single operations, 40x for built-ins and 5x for ETS. The 1.0 table met them;
the 3.0.0 candidate is mixed: function call at 38.3x is above 35x,
lists:reverse/1at 39.7x is just under 40x, ETS at 7.66x is above 5x, and closures at 243x remain the known miss. The rows are kept visible rather than quietly dropped.
An interpreter optimisation was not attempted for 1.0. The table says where the work is — closures, then ETS — and doing it from this data is the first task after the release, not part of it.