SLeeLa SLeeLa # Regression Corpus Deterministic hostile and malformed inputs, each with a documented outcome. A case exists because the input once exposed a defect or because it protects a contract the code must keep. The corpus is run by `python/test_corpus.py` as the `corpus` group of `python/run_regression.py`, so it runs with the rest of the regression set (also with `--fast`), in CI, and through `test-suites/run-all.sh --regression`. ## Rules - **Bounded and deterministic.** Every case finishes within seconds. A hang is a failure. Random input uses a fixed seed (see `c/reg_http6.c`); nothing reads the network or the clock. - **No crash, ever.** Whatever a case expects, the tool must not die from a signal and must not exit with a shell failure code (126 or above). A crash on hostile input is always a defect. - **No secrets or personal data.** Fixtures are synthetic. - **The expected outcome is written beside the fixture**, in a `NAME.expect` sidecar, and again in the table below. A test fails if a fixture has no `.expect`, if a case is missing from the table, or if a `reference` path does not exist. - **Every case names its reason**: the `reference` file it protects, and for a former defect the `defect` id it was written for. ## Layout corpus/ README.md this file (the case table is checked against the fixtures) annotations/ NAME.sleela + NAME.expect document annotations, @next destinations parser/ NAME.sleela + NAME.expect lexer, version pragma, parser and semantic rejection http/ NAME.hex + NAME.expect HTTP 6.0 frames as annotated hex text tools/http6_decode.c driver that feeds one hex fixture to http6_frame_decode Inputs too large to commit (a 2,000,000-character identifier, 20,000 nested parentheses) are generated by the `GENERATED` list in `python/test_corpus.py`; they have the same fields and appear in the same table. ## The `.expect` format One `key: value` per line; `#` starts a comment; a key may repeat. | Key | Meaning | |---|---| | `id` | Unique case id (`FAMILY-WHAT`). Used in test names and in the table. | | `family` | One of the families below. | | `expect` | `accept` or `reject`. | | `command` | `check` (default) or `run`; the `sleela` subcommand used. Ignored for HTTP cases. | | `message` | For `reject`: text that must appear in the diagnostics. Repeatable. | | `warning` | For `accept`: text that must appear on stderr as a warning. Repeatable. | | `output` | For `accept` with `run`: text that must appear in the program output. Repeatable. | | `rc` | HTTP cases only: the exact result of `http6_frame_decode`. | | `status` | `known-issue` when the case states a contract the code does not honour yet. | | `reference` | The source or document the case protects (first word must be an existing path). | | `defect` | The id of the defect the case was written for, if any. | | `note` | One or two sentences: what the input is and why it matters. | A `known-issue` case is reported as `OPEN` at the end of the run and does not fail it. When the code starts to honour the contract the report says `FIXED`; remove the `status` line to make the case a hard assertion. ## Families | Family | What it covers | Where | |---|---|---| | `annotations` | Document annotation syntax, repetition policy, unknown names, the `@next` forwarding rules. | `annotations/` | | `traversal` | `@next` destinations that would leave the architecture tree: absolute, parent (`..`), embedded parent, drive letters, backslashes. | `annotations/` | | `percent-encoding` | Encoded traversal (`%2e%2e`). The policy refuses `%` instead of trusting every consumer to decode safely. | `annotations/` | | `http-framing` | HTTP 6.0 frames: truncated header, truncated payload, oversized and maximal lengths, wrong version, undefined types. | `http/` | | `parser-rejection` | Version pragma, lexical, syntax and semantic errors; each must be a clear diagnostic. | `parser/` | | `parser-acceptance` | Inputs that look odd but are legal. | `parser/` | | `resource-limits` | Compile-time nesting, chain and identifier limits, and run-time recursion, array and stack limits. All end in a diagnostic, not a crash. | generated | ## Limits the corpus pins The compiler refuses, with a `Syntax error ... too deep / too long` message, source that would otherwise exhaust the native stack: more than 1000 levels of `()`, `[]` and `{}`, more than 5000 operators between two statement separators, more than 1000 prefix operators in a row, and more than 1000 back-to-back casts. The caps sit far above real programs; `LIMIT-NEST-AT-CAP` and `LIMIT-CHAIN-AT-CAP` run programs just under them. Statements nest through their bodies and through `else` (an else-if chain is one level per link), with or without braces: more than 3000 levels is refused (`LIMIT-ELSE-IF-CHAIN`, `LIMIT-NESTED-LOOPS-NO-BRACES`, `LIMIT-NESTED-DO-NO-BRACES`; `LIMIT-ELSE-IF-AT-CAP` runs 2,900 links). Member-access (`.`) and label (`:`) chains count with the operators (`LIMIT-MEMBER-CHAIN`, `LIMIT-LABEL-CHAIN`). A crash from a shape not listed is a defect to add here. ## Known gaps None open. `ANN-CLASS-BOUND` and `ANN-NEXT-SPELLING` were known issues and are now hard assertions: a `@next` written before a class, method or field is judged by the same destination rule as a document annotation, and the forwarder reports a destination as written (`system/net`, not `system / net`). `impl/frontend/ANNOTATION_PIPELINE.md` describes the `@name(value)` form the parser accepts. ## Running python3 test-suites/regression/python/run_regression.py --only corpus # the corpus python3 test-suites/regression/python/run_regression.py --only corpus -k ANN # cases whose id contains ANN python3 test-suites/regression/python/run_regression.py --only corpus -v # one line per case ## Adding a case 1. Put the input in `corpus//NAME.sleela` (or `.hex`), or add a `gen(...)` entry for a large one. 2. Write `NAME.expect` with at least `id`, `family`, `expect`, `reference` and `note`. 3. Run the corpus. A new case that exposes a defect should fail first; fix the code, then keep the case. 4. Add the id to the table below. The bookkeeping test names any case that is missing. ## Cases | ID | Family | Input | Outcome | What it protects | |---|---|---|---|---| | `ANN-AFTER-CLASS` | annotations | `annotations/annotation-after-declaration.sleela` | reject (`check`) | A document annotation after the first class is refused. | | `ANN-DUP-SCOPE` | annotations | `annotations/scope-duplicate.sleela` | reject (`check`) | Only next, requires, capability, counter and supporter may repeat. | | `ANN-NEXT-BARE` | annotations | `annotations/next-bare-marker.sleela` | reject (`check`) | @next without parentheses carries no destination. | | `ANN-NEXT-EMPTY` | annotations | `annotations/next-empty-value.sleela` | reject (`check`) | A value-form annotation needs a value. | | `ANN-NEXT-MULTI-CHECK` | annotations | `annotations/next-multiple-checks.sleela` | accept (`check`) | Several @next declarations are legal metadata; only forwarding refuses them (next case). | | `ANN-NEXT-MULTI-RUN` | annotations | `annotations/next-multiple-forwarding.sleela` | reject (`run`) | The runtime refuses ambiguous forwarding before any code runs. | | `ANN-NEXT-SPELLING` | annotations | `annotations/forwarded-destination-keeps-its-slashes.sleela` | accept (`run`) | The parser joins annotation tokens with spaces; the forwarder now closes the gaps around '/', '.' and '-', so the destination is reported as written. | | `ANN-NEXT-SPELLING-PUNCT` | annotations | `annotations/spaced-destination-words-stay-apart.sleela` | accept (`run`) | Hyphens and dots inside a destination survive the spacing repair. | | `ANN-OK-DOCUMENT` | annotations | `annotations/ok-document-annotations.sleela` | accept (`check`) | The reference shape: document annotations, one safe relative @next. | | `ANN-UNKNOWN` | annotations | `annotations/unknown-annotation.sleela` | accept (`check`) | Unknown annotations are kept and reported as a warning, never silently dropped. | | `ANN-UNTERMINATED` | annotations | `annotations/unterminated-parenthesis.sleela` | reject (`check`) | The parenthesis is never closed. | | `ANN-CLASS-BOUND` | traversal | `annotations/class-bound-annotation-is-not-document-metadata.sleela` | reject (`check`) | An annotation written directly before a class binds to that class, not to the document. The same destination rule applies to it (it used to bypass... | | `ANN-FIELD-BOUND` | traversal | `annotations/field-bound-next-absolute.sleela` | reject (`check`) | A field-level @next is judged by the same destination rule. | | `ANN-METHOD-BOUND` | traversal | `annotations/method-bound-next-parent.sleela` | reject (`check`) | A method-level @next is judged by the same destination rule. | | `ANN-NEXT-ABSOLUTE` | traversal | `annotations/next-absolute-path.sleela` | reject (`check`) | An absolute destination would leave the architecture tree. | | `ANN-NEXT-BACKSLASH` | traversal | `annotations/next-backslash.sleela` | reject (`check`) | A backslash is not a legal source character; the lexer refuses it before the policy sees it. | | `ANN-NEXT-DRIVE` | traversal | `annotations/next-drive-letter.sleela` | reject (`check`) | Windows-absolute destination. | | `ANN-NEXT-EMBEDDED-PARENT` | traversal | `annotations/next-embedded-parent.sleela` | reject (`check`) | A parent component in the middle of a path. | | `ANN-NEXT-PARENT` | traversal | `annotations/next-parent-traversal.sleela` | reject (`check`) | Parent-directory traversal. | | `ANN-NEXT-PERCENT` | percent-encoding | `annotations/next-percent-encoded-parent.sleela` | reject (`check`) | Percent-encoded parent directory. | | `HTTP6-EMPTY` | http-framing | `http/empty-input.hex` | reject, decode returns -2 | Nothing to read is a short header, not a crash. | | `HTTP6-LENGTH-FFFFFFFF` | http-framing | `http/maximum-length-field.hex` | reject, decode returns -3 | The largest 32-bit length must not wrap a size computation. | | `HTTP6-OK-DATA` | http-framing | `http/data-with-payload.hex` | accept, decode returns 0 | Header fields and payload round-trip. | | `HTTP6-OK-PING` | http-framing | `http/ping-empty-payload.hex` | accept, decode returns 0 | The smallest legal frame. | | `HTTP6-OVERSIZE` | http-framing | `http/oversized-length.hex` | reject, decode returns -3 | One byte over the 16 MiB cap. | | `HTTP6-TRUNC-HEADER` | http-framing | `http/truncated-header.hex` | reject, decode returns -2 | One byte short of a header. | | `HTTP6-TRUNC-PAYLOAD` | http-framing | `http/truncated-payload.hex` | reject, decode returns -3 | The declared length exceeds the bytes available. | | `HTTP6-TYPE-0` | http-framing | `http/type-zero.hex` | reject, decode returns -3 | Types start at 1. | | `HTTP6-TYPE-21` | http-framing | `http/type-past-last.hex` | reject, decode returns -3 | Type 20 (RECIPIENT_LABEL) is the last one. | | `HTTP6-VERSION` | http-framing | `http/wrong-version.hex` | reject, decode returns -3 | Only version 6 is accepted. | | `PARSE-BRACE-MISSING` | parser-rejection | `parser/missing-closing-brace.sleela` | reject (`check`) | The file ends inside a method. | | `PARSE-COMMENT-UNTERMINATED` | parser-rejection | `parser/unterminated-block-comment.sleela` | reject (`check`) | The << comment has no >>. | | `PARSE-EMPTY` | parser-rejection | `parser/empty-file.sleela` | reject (`check`) | A zero-byte file. | | `PARSE-INT-OVERFLOW` | parser-rejection | `parser/integer-literal-overflow.sleela` | reject (`check`) | A literal wider than 64 bits. | | `PARSE-NO-CLASSES` | parser-rejection | `parser/no-classes.sleela` | reject (`check`) | A pragma and nothing else. | | `PARSE-NUL` | parser-rejection | `parser/nul-byte-in-source.sleela` | reject (`check`) | A NUL byte inside code is a lexical error. | | `PARSE-PRAGMA-FUTURE` | parser-rejection | `parser/pragma-too-new.sleela` | reject (`check`) | One minor above the ceiling. | | `PARSE-PRAGMA-MALFORMED` | parser-rejection | `parser/pragma-malformed.sleela` | reject (`check`) | A non-numeric version. | | `PARSE-PRAGMA-OLD` | parser-rejection | `parser/pragma-too-old.sleela` | reject (`check`) | Below the floor. | | `PARSE-PRAGMA-OVERFLOW` | parser-rejection | `parser/pragma-huge-number.sleela` | reject (`check`) | A major number that overflows an int is malformed, not wrapped. | | `PARSE-STRING-UNTERMINATED` | parser-rejection | `parser/unterminated-string.sleela` | reject (`check`) | The closing quote is missing. | | `SEM-DUP-METHOD` | parser-rejection | `parser/duplicate-method.sleela` | reject (`check`) | Two methods with one signature. | | `SEM-UNDECLARED` | parser-rejection | `parser/undeclared-variable.sleela` | reject (`check`) | Assignment before declaration. | | `SEM-UNKNOWN-TYPE` | parser-rejection | `parser/unknown-type.sleela` | reject (`check`) | Type check, not parse. | | `PARSE-OK-BLANK-LINES` | parser-acceptance | `parser/blank-and-whitespace-lines.sleela` | accept (`run`) | Empty and whitespace-only lines before the first declaration. | | `PARSE-OK-COMMENTS` | parser-acceptance | `parser/every-comment-form.sleela` | accept (`run`) | All four comment forms are skipped; an angle comment must start its line. | | `PARSE-OK-CRLF` | parser-acceptance | generated | accept (`run`) | A file with Windows line endings is the same program. Generated because the repository normalizes text files to LF. | | `PARSE-OK-MAX-LONG` | parser-acceptance | `parser/largest-long-literal.sleela` | accept (`run`) | The largest 64-bit literal is in range (one more is PARSE-INT-OVERFLOW). | | `PARSE-OK-NO-FINAL-NEWLINE` | parser-acceptance | `parser/no-trailing-newline.sleela` | accept (`run`) | The last token may end the file. | | `PARSE-OK-UNICODE` | parser-acceptance | `parser/unicode-string-literal.sleela` | accept (`run`) | Non-ASCII text, including a character outside the BMP, survives lexing and printing. | | `LIMIT-CAST-CHAIN` | resource-limits | generated | reject (`check`) | 30,000 back-to-back casts nest without any open bracket. | | `LIMIT-CHAIN-AT-CAP` | resource-limits | generated | accept (`run`) | A 4,000-operator statement is under the cap and still runs. | | `LIMIT-ELSE-IF-AT-CAP` | resource-limits | generated | accept (`run`) | 2,900 else-if links is under the cap and runs. | | `LIMIT-ELSE-IF-CHAIN` | resource-limits | generated | reject (`check`) | 60,000 chained else-if links. Each link is one level of statement recursion with no bracket to count. | | `LIMIT-LABEL-CHAIN` | resource-limits | generated | reject (`check`) | 60,000 stacked statement labels. | | `LIMIT-LONG-IDENTIFIER` | resource-limits | generated | accept (`check`) | A 2,000,000-character identifier is legal. Dependency discovery used std::regex, which recurses per character and crashed. | | `LIMIT-LONG-IDENTIFIER-RUN` | resource-limits | generated | accept (`run`) | A 50,000-character identifier runs end to end. | | `LIMIT-MANY-CLASSES` | resource-limits | generated | accept (`check`) | 20,000 declarations. | | `LIMIT-MEMBER-CHAIN` | resource-limits | generated | reject (`check`) | A 60,000-call member chain builds a left-deep tree. | | `LIMIT-MEMBER-CHAIN-AT-CAP` | resource-limits | generated | accept (`run`) | A 4,000-call member chain is under the cap and runs. | | `LIMIT-NEST-AT-CAP` | resource-limits | generated | accept (`run`) | 900 nested parentheses is under the cap and still runs. | | `LIMIT-NEST-BLOCKS` | resource-limits | generated | reject (`check`) | 5,000 nested blocks. | | `LIMIT-NEST-PARENS` | resource-limits | generated | reject (`check`) | 20,000 nested parentheses. The parser recursed once per level and crashed with a segmentation fault. | | `LIMIT-NESTED-DO-NO-BRACES` | resource-limits | generated | reject (`check`) | 20,000 nested do statements. | | `LIMIT-NESTED-LOOPS-NO-BRACES` | resource-limits | generated | reject (`check`) | 60,000 nested for statements without braces. | | `LIMIT-OPERATOR-CHAIN` | resource-limits | generated | reject (`check`) | One statement of 100,000 chained operators builds a left-deep tree that later passes walk recursively. | | `LIMIT-PREFIX-RUN` | resource-limits | generated | reject (`check`) | 5,000 prefix operators in a row. | | `LIMIT-STRING-1MB` | resource-limits | generated | accept (`run`) | A one-megabyte string literal. | | `RUN-ARRAY-NEGATIVE` | resource-limits | generated | reject (`run`) | A negative array size. | | `RUN-ARRAY-TOO-LARGE` | resource-limits | generated | reject (`run`) | An array far beyond the heap limit. | | `RUN-DIVIDE-BY-ZERO` | resource-limits | generated | reject (`run`) | Integer division by zero. | | `RUN-MANY-LOCALS` | resource-limits | generated | reject (`run`) | 30,000 locals in one method exhaust the VM stack with a runtime error. | | `RUN-RECURSION` | resource-limits | generated | reject (`run`) | Unbounded recursion ends with a runtime error, not a crash. |