Home /Modules/docs/SLEELA.PERFORMANCE.md

SLEELA.PERFORMANCE.md

docs/reference/SLEELA.PERFORMANCE.md · 303 line(s) · markdown

<img align="right" src="../../images/sleela/sleela-medium-profile-permanent-icon-002.jpg" width="45" height="45" alt="SLeeLa">

SLeeLa Performance Report

Report edition: 001 Repository: mearvk/SLeeLa Baseline: SLeeLa Syntax 3.0 Last consolidated: 2026-11-07 Status: Arithmetic datasets recorded and checksum-validated; VM/language conformance passed on all three operating systems; native portability remains blocked by regression-test harness failures (2026-11-07)

1. Executive summary

This report consolidates the arithmetic performance measurements currently recorded for native C11, native C++17, and Java 21+, and tracks the corresponding SLeeLa VM qualification work.

The report now contains two separately identified native arithmetic datasets: the earlier local Linux reference run and a hosted GitHub Actions Linux run. Neither should be treated as a universal language ranking or release-grade measurement.

The corrected integrated workflow measured all twelve runnable SLeeLa VM targets across integer and double addition, subtraction, multiplication, and division. All 96 target-operation cases and all 480 sample checksum validations passed. The overall VM workflow later failed only because the checked-in Standard VM Modium manifests were stale; that later registry freshness check does not invalidate the completed arithmetic artifact.

2. Current qualification status

EnvironmentArithmetic dataExecution evidenceStatus
Native C11int widths, float32, float64; signed 128-bit extensionLocal reference + hosted CI run 42340090666Recorded; both suites passed
Native C++17int widths, float32, float64; signed 128-bit extensionLocal reference + hosted CI run 42340090666Recorded; both suites passed
Java 21+integer-width emulations, float32, float64; no primitive 128-bitLocal reference + hosted CI run 42340090666Recorded; both suites passed
SLeeLa Basic Native VMint, double; +, -, *, /Workflow run 42342660507PASS — 8 cases × 5 samples; checksums validated
SLeeLa Premium SLVM 1–11int, double; +, -, *, /Workflow run 42342660507PASS — 11/11 targets; 88 cases × 5 samples; checksums validated

Workflow checked: Complete SLeeLa VM and Language Conformance — run 42340090666. The run completed with 4 of 5 jobs successful. The arithmetic speed job passed; the VM job failed after the twelve targets were runnable and the Premium self-checks passed, when the new benchmark runner attempted to read an undefined checks variable. That defect is fixed in the current source. The VM timing results from this first attempt remain unavailable.

Latest cross-platform CI qualification — 2026-11-07

The corrective workflows are still running at the time of this report update. Their current state is recorded as a snapshot, not a final pass/fail determination:

Basic Native VM + Premium SLVM 1-11: in_progress Arithmetic speed — C11, C++17, Java 21+: completed (success) C11, C++17, Java 21+ (macos-latest): completed (success) C11, C++17, Java 21+ (ubuntu-latest): completed (success) C11, C++17, Java 21+ (windows-latest): in_progress

Native runtime (ubuntu-latest): in_progress Native runtime (macos-latest): in_progress Native runtime (Windows 10+ / MinGW-w64): in_progress

The previously validated arithmetic dataset remains: SLeeLa Basic Native plus Premium SLVM 1–11, 96/96 target-operation cases and 480/480 sample checksum validations; C11/C++17/Java 21+ comparator suite, 128/128 cases on the recorded hosted Linux run. The in-progress reruns are intended to close the separate Modium-manifest and native-portability gaps. Do not mark cross-platform VM conformance or native portability complete until these runs finish successfully. This status snapshot should be replaced with final job conclusions after completion.

Final corrective CI results — 2026-11-07

The two corrective workflows have now completed.

These failures are test-harness portability issues observed after the earlier native build and several OSLS checks succeeded. They do not overturn the successful VM/language conformance workflow or the separately validated arithmetic measurements. Native portability is NOT qualified on any of the three operating systems until the shell recipe and Windows path expectation are corrected and the workflow is rerun.

3. Native C11 / C++17 / Java 21+ arithmetic results

The following values are the median nanoseconds per operation (ns/op) from Arithmetic Speed Series — Iteration 001. Lower values indicate lower observed time for this particular benchmark path.

TypeOperatorC11C++17Java 21+
i8+0.5750.6030.824
i8-1.1281.1731.382
i8*1.2411.2765.483
i8/1.7041.7327.171
u8+0.5930.5701.075
u8-1.2200.8351.164
u8*1.2810.9791.798
u8/1.1161.1432.476
i16+0.5670.6031.259
i16-1.1211.1641.364
i16*1.2641.1911.818
i16/1.7681.7292.251
u16+0.6620.6451.102
u16-1.1751.1381.121
u16*1.1621.2391.026
u16/1.1601.1541.985
i32+0.6630.6791.158
i32-1.2300.9871.510
i32*1.2621.0131.218
i32/1.7541.6932.017
u32+0.7070.7360.602
u32-1.2441.1911.230
u32*1.4211.3170.966
u32/1.7101.8272.001
i64+0.5870.6870.708
i64-1.1801.2541.110
i64*1.4171.4260.992
i64/2.0872.2232.010
u64+0.5770.6980.573
u64-1.2091.1700.728
u64*1.3271.3170.827
u64/1.9662.0431.933
i128+0.6100.641N/A
i128-0.8620.798N/A
i128*1.0200.932N/A
i128/2.4313.691N/A
f32+1.2431.1420.813
f32-1.2611.2560.839
f32*1.2931.2360.867
f32/1.0190.9670.911
f64+1.1281.1310.782
f64-1.2501.2230.848
f64*1.2241.2910.842
f64/1.3561.4101.318

3.1 Native reference host and method

PropertyRecorded configuration
OSLinux 6.18.57 x86_64, glibc 2.41
CPUAMD EPYC 9V74; container allocation of 3 logical CPUs
RAM visible to container6,091,004 kB
C compilerGCC 14.2.0
C++ compilerG++ 14.2.0
JavaOpenJDK 21.0.12.1
Native optimization-O2
Operations per case2,000,000
Operand set4,096 deterministic elements; seed 0x53435350
Repetitions5 measured samples per type/operator
Warm-upOne untimed native process; three Java warm-up sweeps inside each measured JVM
Timing sourceMonotonic high-resolution clock
Reported statisticMedian ns/op

The native benchmark exercises indexed operands and checksum folding. It measures the full loop/index/operand/arithmetic/checksum path, not an isolated processor instruction.

3.2 Type and language qualifications

3.2 Hosted GitHub Actions Linux arithmetic results

Execution: Complete SLeeLa VM and Language Conformance — run 42340090666, arithmetic-speed job. Artifact: comparison-arithmetic-speed-results.json, artifact ID 13561852004. Result: PASS — 128/128 measurements/checksum cases (C11 44/44, C++17 44/44, Java 21+ 40/40). Each case used five measured samples; the values below are the medians in ns/op.

Hosted runner metadata: Linux 7.0.0-1019-azure, x86_64, glibc 2.43, 4 logical cores. The CPU model was not exposed by the runner metadata. These hosted values are a separate dataset from the local reference values in section 3 and must not be merged or averaged with them.

TypeOperatorC11C++17Java 21+
f32*1.3741.6594.881
f32+1.1521.1581.245
f32-1.2481.4071.288
f32/1.5201.5173.321
f64*1.4411.1531.006
f64+1.4391.1531.018
f64-1.4411.1511.107
f64/1.3871.2871.469
i128*1.2501.247N/A
i128+0.9200.914N/A
i128-1.1510.952N/A
i128/3.1783.463N/A
i16*0.9701.1511.822
i16+0.6710.6781.068
i16-0.8741.1551.781
i16/1.7281.7293.191
i32*1.1511.0811.133
i32+0.6750.6710.734
i32-1.1510.8951.117
i32/1.7301.7302.887
i64*1.1521.1530.968
i64+0.6410.7760.615
i64-0.8661.1511.100
i64/2.8782.8832.887
i8*0.9941.1616.466
i8+0.6960.6750.891
i8-0.9081.1601.344
i8/1.7421.73913.812
u16*1.0531.1531.536
u16+0.6730.6731.063
u16-1.0571.1531.344
u16/2.0162.0172.950
u32*0.9471.1520.819
u32+0.6710.7520.619
u32-0.8641.1510.621
u32/1.7281.7262.888
u64*1.1511.1130.852
u64+0.6440.6880.617
u64-1.1820.9010.630
u64/2.8822.8792.883
u8*1.1561.0851.518
u8+0.6740.6731.769
u8-1.1600.8720.979
u8/1.7311.7332.981

The hosted runner and local reference benchmark are distinct executions with different hosts and timing environments. Compare only within each dataset unless a controlled matched-runner protocol is introduced.

4. SLeeLa VM arithmetic qualification

The SLeeLa-specific workload is defined in:

The workload measures SLeeLa int and double for addition, subtraction, multiplication, and division on twelve targets: Basic Native and Premium SLVM 1 through 11.

4.1 Timing boundary

The benchmark uses the SLeeLa monotonic nanosecond timer around the measured loop. Process launch, CLI startup, compilation/build, VM selection, Premium admission, and per-operation warm-up are outside the measured interval. Each operation has a 50,000-iteration warm-up and a 250,000-iteration measured loop, with five executions per target.

The measured loop includes VM loop-counter, branch, assignment, and interpreter-dispatch overhead. It is an end-to-end in-VM workload, not isolated arithmetic instruction latency.

4.2 SLeeLa results matrix

The hosted CI measurements are now complete. See section 4.3 for the validated medians and the exact workflow artifact. All twelve targets passed all eight arithmetic cases.

4.3 Hosted VM arithmetic results — validated

Workflow: Complete SLeeLa VM and Language Conformance — run 42342660507 Commit: ff75886e08bbff7f325d1ca84b0e86b930bda54c Artifact: sleela-complete-vm-matrix — contains comparison-sleela-vm-arithmetic-results.json. Arithmetic status: PASS — 12/12 targets, 96/96 target-operation cases, 480/480 sample checksums.

The hosted Linux runner reported Linux 7.0.0-1019-azure, x86_64, 4 logical cores. Each target ran eight cases (int and double × +, -, *, /), with 250,000 measured iterations per case, 50,000 warm-up iterations, and five samples. Values below are median nanoseconds per operation. The integer checksums were exact; double checksums were validated at a 1e-6 resolution after correcting lossy default double-to-text formatting. This is a distinct in-VM workload from the native-language matrix in section 3; do not treat the figures as directly comparable without matching harnesses and timing boundaries.

Targetint +int -int *int /double +double -double *double /
Basic Native72.10772.17872.11272.26080.55279.12479.76882.920
Premium SLVM 172.53772.32172.51972.48580.88179.68280.46583.057
Premium SLVM 272.54272.27772.24172.41080.65679.70279.91083.378
Premium SLVM 372.56672.51872.52072.32380.48679.80979.85483.057
Premium SLVM 472.57172.49872.21872.29780.57979.71079.97182.897
Premium SLVM 573.07472.51572.18772.25580.51879.74679.88982.993
Premium SLVM 672.80772.65072.43672.31880.73879.55479.91683.165
Premium SLVM 772.71872.18572.60772.30980.95479.65679.95083.103
Premium SLVM 875.77975.11175.32975.53082.88881.71782.72184.797
Premium SLVM 972.80572.52072.29772.23780.58179.73479.74983.080
Premium SLVM 1072.57672.34072.23872.33280.50679.60880.11983.183
Premium SLVM 1181.84281.70681.85382.46887.85887.16587.60691.017

Qualification note: all twelve runnable targets completed every arithmetic case and checksum check. The overall VM workflow was nevertheless marked failed after the arithmetic suite passed, because the subsequent tools/generate-modium.py --check step found the checked-in MODIUM.json and rebuild.sql artifacts stale for Standard VM generations 001–010. That is a generated-registry freshness failure, not an arithmetic or VM execution failure. The arithmetic artifact is valid and retained above.

5. Memory and RAM performance

The arithmetic results above do not measure RAM latency or bandwidth. The host's visible RAM figure is environment metadata, not a memory-speed result.

A separate memory benchmark is required to report:

Memory results must identify the data size, access pattern, repetitions, warm-up, thread count, and timing boundary. They must not be mixed into arithmetic ns/op tables.

6. Boolean and logical-operation performance

Boolean values are excluded from arithmetic. A separate suite should measure logical AND, OR, NOT, equality/inequality, and short-circuit behavior. It should validate truth tables and side-effect behavior before accepting timing data.

7. Interpretation and limitations

  1. The native-language measurements are a single local Linux-container reference run, not a controlled cross-platform benchmark.
  2. Hosted CI measurements, when available, should be recorded separately with runner OS, CPU, compiler/JDK versions, and run identifiers.
  3. The C/C++/Java and SLeeLa VM suites use different harness paths and operation counts. Direct ranking should wait until a deliberately matched workload and timing model are established.
  4. This report does not claim a universal fastest language or VM.
  5. No memory bandwidth, random RAM latency, cache sweep, allocation, or garbage-collection performance is included in the arithmetic data.
  6. SLeeLa VM measurements are available for the eight tested int/double cases; other integer widths, float32, memory access, and Boolean logic remain unmeasured for SLeeLa.

8. Source reports and reproduction

Native-language reference reproduction:

python3 comparison/tests/run_arithmetic_speed.py

SLeeLa VM reproduction after the Basic Native and Premium VMs are built:

python3 comparison/tests/run_sleela_vm_arithmetic_speed.py

The SLeeLa runner writes comparison-sleela-vm-arithmetic-results.json; the integrated workflow retains it in the sleela-complete-vm-matrix artifact. The validated run is 42342660507 (artifact 13562815822).


Current conclusion: C11, C++17, and Java 21+ have local reference data and a completed hosted Linux arithmetic dataset (128/128 cases passed). SLeeLa Basic Native and Premium SLVM 1–11 now have validated arithmetic measurements (96/96 target-operation cases; 480/480 checksum samples passed). The overall VM workflow remains red because the later Standard VM Modium freshness check detected stale generated files. RAM performance, broader SLeeLa numeric widths, float32, and Boolean logic still require separate benchmark suites.

9. Common eight-case comparison — Iteration 002

A consolidated table now places the eight shared hosted-Linux cases (i64 and f64 × +, -, *, /) side by side for C11, C++17, Java 21+, Basic Native, and Premium SLVM 1–11: Arithmetic Comparison — Common Eight Cases. This is a common-coverage view only. The source datasets still use different iteration counts, operand patterns, and timing boundaries, so cross-language and native-versus-VM timing ratios remain invalid. A genuinely equivalent comparison requires a new matched-protocol CI run.