Home /Modules/comparison/CONFORMANCE-MATRIX.md

CONFORMANCE-MATRIX.md

comparison/CONFORMANCE-MATRIX.md · 85 line(s) · markdown

Conformance Matrix

Standard SCS-MATRIX-001 | Edition 1.1 Status: baseline and batch 002 passed; batch 003 array/string suite pending CI; feature qualification remains scoped.

1. Legend

Native/defined = language/runtime model; Managed = managed runtime; Library/API = library/platform facility; Explicit contract = must be defined and tested; Varies = target-dependent; Separate path = requires explicit interoperability.

2. Capability comparison

CapabilitySLeeLaJavaCC++
ExecutionSLeeLa VM / selected targetJVMNativeNative
OutputSLeeLa artifact/IRJVM class filesNative objectNative object
Integer widthsSyntax 3.0 width families; verify each implementationFixed primitive widthsFundamental widths vary; exact-width types optionalFundamental widths vary; exact-width types optional
BooleanSLeeLa contractboolean_Boolbool
Floating pointVM contractJava rulesStandard + implementationStandard + implementation
StringsSLeeLa library/runtimeStringExplicit encoding conventionLibrary/encoding convention
BoundsVM/runtime contractJava array checksRaw arrays uncheckedRaw arrays unchecked
ReclamationSLeeLa runtime contractGarbage collectionExplicit/manual and automatic storageObject lifetime/RAII and explicit allocation
Deterministic destructionVerify resource contractNot guaranteed by GCStorage-duration rules and explicit releaseScope-based object destruction
ExceptionsSLeeLa contractBuilt-inNo standard language exceptionsBuilt-in
ConcurrencySLeeLa runtime contractJava Memory ModelC memory modelC++ memory model
Native ABIStable C boundary intendedJNI/native interfaces if usedPlatform C ABICompiler/platform C++ ABI
Java interopExternal Java path documented; embedded JVM separateJVM itselfExplicit JNI/IPCExplicit JNI/IPC

3. Repository architecture notes

Repository documentation describes SLeeLa Syntax 3.0 as current; C ABI as preferred native boundary with C++ orchestration behind it; Java interoperability as external process delegation by default; embedded JVM as not implemented in the documented default configuration; and /lib/java as a SLeeLa source/binding inventory, not a Java implementation.

These are architecture/documentation statements, not proof that every feature or platform is qualified.

4. Qualification checklist

AreaEvidence requiredCurrent status
Syntax/parserVersioned compiler tests/logsNot assessed by this comparison suite
Primitive representationRuntime tests and ABI probesNot assessed
Arithmetic / basic control flow / ASCII outputShared baseline vectorsPASS — Ubuntu, macOS, Windows; SLeeLa VM on Ubuntu
Fixed-width integer edges / UTF-8 byte and code-point countsSCS-TEST-002 edge vectorsPASS — C11, C++17, Java on Ubuntu, macOS, Windows; SLeeLa VM not yet covered
Arrays/strings beyond ASCIISCS-TEST-003 array bounds and strict UTF-8 vectorsPending CI — C11, C++17, Java; SLeeLa VM not yet covered
Memory ownershipRuntime and native sanitizer testsNot assessed
C ABIHeader consumer tests across compilersNot assessed
C++ wrapperException containment and ABI testsNot assessed
Java processProbe, arguments, streams, exit testsNot assessed
JNI/embedded JVMLifecycle, references, threads, shutdownNot assessed
Cross-platform C/C++/Java baselineUbuntu, Windows, macOS workflow artifactsPASS — all three OS jobs
SLeeLa VM baselineSyntax 3.0 VM execution artifactPASS — build and vector on Ubuntu
PerformanceReproducible benchmark reportNot assessed

5. Corrected CI evidence

Latest baseline run: 42331591283, conclusion success. Numeric/Unicode batch 002: 42332366905, conclusion success, commit 087b9c2d77d0a32cf345c9ec6289a98ea0da00dd.

JobResult
Shared vectors — UbuntuPASS
Shared vectors — macOSPASS
Shared vectors — WindowsPASS
SLeeLa Syntax 3.0 VM vectors — UbuntuPASS

Two additional corrected runs also completed successfully: 42331589662 and 42331561514.

The initial run 42331126707 failed for two diagnosed reasons: Windows Java tool discovery under MSYS2 Python, and whole-stdout comparison that treated SLeeLa runtime diagnostics as application output. The workflow now uses native Windows Python with Java 21 and the UCRT64 compiler path; the SLeeLa validator checks the final three nonblank application-output lines. The corrections passed in the completed runs above.

See RUNNING.PASS.JAVA.001.md for baseline evidence and RUNNING.PASS.JAVA.002.md for the numeric/Unicode edge-vector evidence and remaining scope limits.

6. Result record

test_suite: repository_commit: sleela_syntax: sleelvac_version: sleela_vm_abi: java_version: c_standard: cpp_standard: compiler_and_version: target_triple: operating_system: build_flags: test_command: passed: failed: skipped: blocked: result_artifact:

Never replace Not assessed with PASS without an actual retained test run. A workflow definition or queued run is not a completed result.