Home /Modules/artificial-intelligence/README.md

README.md

artificial-intelligence/character-recognition/README.md · 130 line(s) · markdown

Character Recognition

A small, dependency-light C11 foundation for recognizing clean, carefully written, pre-segmented European characters and producing a reviewable document transcript.

What it does

Scope and important limits

This is a character classifier, not a complete page-scanning OCR system. The caller must segment a page into individual glyph crops and supply their reading order, spaces, and line breaks. The first version is designed for separated, clean glyphs. Connected cursive writing, overlapping strokes, automatic word/line segmentation, page deskew, and language-model correction are not implemented. A connected handwritten word must not be passed as one glyph and expected to decode correctly.

The nearest-neighbour score is a normalized pixel-similarity measure, not a statistical probability or legal confidence rating. The threshold must be calibrated against representative handwriting samples. Register multiple examples per writer and character; a single alphabet specimen is not adequate for reliable archival transcription.

Document preservation

The writer creates two separate outputs:

  1. A plain UTF-8 transcript for search and reading.
  2. A JSONL review sidecar that records the source reference and each character's recognition result.

It never opens or modifies the source scan. Preserve the original scan as the authoritative source; compare and human-review the transcript before relying on it in legal work. This prototype does not provide a digital signature, chain-of-custody guarantee, or evidentiary certification.

C interface

Include character_recognition.h and compile character_recognition.c as C11. Create a model, add labelled normalized 28×28 glyphs with slcr_model_add, normalize incoming grayscale crops with slcr_normalize_glyph, and classify them with slcr_model_recognize.

For a document, construct an ordered slcr_document_glyph array. Set space_before or line_break_before from the page segmentation stage, then call slcr_write_document with distinct transcript and review-sidecar paths.

Page segmentation, reusable models, and review corrections

The C API now includes a dependency-free PGM page reader (P2 and P5, up to 8-bit grayscale) and a simple projection-based page segmenter for binary page images. Load the original grayscale page with slcr_load_pgm, apply a threshold to make a binary foreground mask, then pass that mask to slcr_segment_page. It groups horizontal ink runs into text lines and vertical ink runs into glyph boxes. The caller crops each box from the original grayscale image and passes it through slcr_normalize_glyph before recognition. The PGM reader does not alter or overwrite the source file. This first segmenter expects clean, separated handwriting; touching cursive glyphs, complex layouts, columns, tables, and automatic page deskew require a stronger layout engine and must be reviewed manually.

slcr_model_save and slcr_model_load persist the labelled glyph bank in a versioned binary format (format v1). The file stores the sample labels and 28×28 pixel patterns; it is intended as a local model asset, not a signed or portable interchange format. Keep model files with their training provenance and writer/sample metadata; do not mix samples from different writers without tracking that fact.

For human review, prepare a UTF-8 TSV correction file with one override per line: zero-based glyph index, a tab, then the corrected single character. Pass it to slcr_write_corrected_document. Positions without an override are recognized again and still obey the configured similarity threshold. Keep the original scan, machine transcript, correction file, and corrected transcript as separate artifacts. This creates an auditable workflow but is not itself a cryptographic chain-of-custody or legal certification system.

Standalone program (main)

A command-line entry point is provided separately in character_recognition_main.c. Keeping main() outside the library source allows applications to link the C API without a duplicate entry point, while also providing a ready-to-run OCR executable.

Build it from the repository root:

make -C tests character-recognition-cli

Run it with a previously trained and saved model:

./tests/character-recognition-cli MODEL.slcrm INPUT.pgm OUTPUT.txt REVIEW.jsonl

Optional arguments are a grayscale threshold (0–255, default 180) and a minimum similarity (0–1, default 0.70):

./tests/character-recognition-cli MODEL.slcrm scan.pgm transcript.txt review.jsonl 180 0.70

Use --help for usage. The executable loads the saved model, reads the PGM page, segments clean separated glyphs, normalizes them, and writes the transcript and JSONL review sidecar. It does not train a model: prepare and validate labelled glyph samples with the library API first. The projection segmenter is intentionally conservative and can merge touching letters or split dotted characters; inspect the sidecar and original scan carefully.

Regression check

From the repository root:

make -C tests ai-character-recognition character-recognition-cli

The standalone Character Recognition C11 GitHub Actions workflow builds the CLI and runs the regression test on Ubuntu, macOS, and Windows/MinGW-w64.