Language-model families: a representative taxonomy
This catalogue provides illustrative examples of model architectures and system patterns relevant to large language models (LLMs). It is a conceptual map for SLeeLa, not a list of commercial model names, checkpoints, or a claim that exactly 2,000โ2,500 distinct architectures exist.
Scope and time horizon
The technically documented families below are primarily those known through 2025. The interval 2026โ2500 is speculative: no one can reliably enumerate future discoveries. Keep forecasts labelled as hypotheses and never mix them with established architectures.
Main families
Statistical language model (n-gram)
- Period: 2000s
- Evidence status: established
- Idea: Predicts tokens from local frequency context; useful baseline, limited long-range structure.
Probabilistic sequence model (hidden-markov-model)
- Period: pre-2010
- Evidence status: established
- Idea: Models hidden states and observed token sequences; historically important for speech and NLP.
Structured prediction (conditional-random-field)
- Period: 2000s
- Evidence status: established
- Idea: Labels sequences using contextual feature dependencies in classical NLP pipelines.
Neural language model (feed-forward-neural-lm)
- Period: 2000s
- Evidence status: established
- Idea: Predicts a token from a fixed context window using dense neural layers.
Recurrent neural network (rnn)
- Period: 2010s
- Evidence status: established
- Idea: Processes sequences with recurrent state; foundational neural sequence architecture.
Gated recurrent network (lstm)
- Period: 2010s
- Evidence status: established
- Idea: Uses input, forget, and output gates to retain information over longer sequences.
Gated recurrent network (gru)
- Period: 2010s
- Evidence status: established
- Idea: A streamlined gated recurrent architecture with update and reset gates.
Encoder-decoder sequence model (seq2seq)
- Period: 2010s
- Evidence status: established
- Idea: Maps an input sequence to an output sequence, often with recurrent or transformer components.
Attention-based encoder-decoder (attention-seq2seq)
- Period: 2010s
- Evidence status: established
- Idea: Allows decoder steps to focus on relevant encoded input positions.
Bidirectional encoder (transformer-encoder)
- Period: 2018 onward
- Evidence status: established
- Idea: Uses self-attention and feed-forward blocks for contextual representations; BERT-style family.
Autoregressive decoder (transformer-decoder)
- Period: 2018 onward
- Evidence status: established
- Idea: Uses causal self-attention to predict the next token; GPT-style family.
Text-to-text transformer (encoder-decoder-transformer)
- Period: 2019 onward
- Evidence status: established
- Idea: Uses an encoder and autoregressive decoder for translation, summarization, and general text tasks.
Conditional-compute model (sparse-mixture-of-experts)
- Period: 2020s
- Evidence status: established
- Idea: Routes tokens or examples to a subset of expert networks to scale capacity with conditional computation.
Retrieval-grounded model system (retrieval-augmented-generation)
- Period: 2020s
- Evidence status: established
- Idea: Combines a language model with search or document retrieval; retrieved evidence is distinct from model weights.
Extended-context model (long-context-transformer)
- Period: 2020s
- Evidence status: established
- Idea: Uses attention variants, memory, or position strategies to process longer sequences.
State-space sequence model (state-space-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Uses structured state-space dynamics or selective scans as an alternative to full attention in some settings.
Hybrid sequence model (hybrid-attention-state-space)
- Period: 2020s
- Evidence status: emerging
- Idea: Combines attention layers with state-space or recurrent components.
Multimodal model (multimodal-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Connects text with image, audio, video, or other modality encoders and decoders.
Audio/speech model (speech-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Processes or generates speech/audio, sometimes through discrete audio tokens or aligned encoders.
Code-specialized model (code-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Trains on source code and text for completion, explanation, transformation, and tool-assisted coding.
Vision-text model (vision-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Aligns visual representations with language for image understanding and visual question answering.
Generative hybrid (diffusion-language-hybrid)
- Period: 2020s
- Evidence status: research
- Idea: Combines language modeling with diffusion-style generation or refinement; implementations vary.
Tool-using model system (agentic-model-system)
- Period: 2020s
- Evidence status: established
- Idea: Combines an LLM with planning, tools, memory, and feedback loops; system design rather than a single neural architecture.
Continual adaptation system (continual-learning-model)
- Period: ongoing
- Evidence status: research
- Idea: Updates or adapts from sequential data while attempting to limit catastrophic forgetting.
Brain-inspired computing model (neuromorphic-language-model)
- Period: ongoing
- Evidence status: research
- Idea: Explores spiking or event-driven computation for language-related workloads.
Compressed deployment variant (quantized-language-model)
- Period: 2020s
- Evidence status: established
- Idea: Uses reduced-precision weights or activations to lower memory and compute; orthogonal to base architecture.
Compressed student model (distilled-language-model)
- Period: 2010s onward
- Evidence status: established
- Idea: Transfers behavior from a larger teacher to a smaller model; orthogonal to base architecture.
Distributed training system (federated-language-model)
- Period: 2020s
- Evidence status: research
- Idea: Trains across decentralized data holders with aggregation and privacy constraints; a training topology, not an architecture.
Hybrid symbolic-neural system (neuro-symbolic-language-system)
- Period: ongoing
- Evidence status: research
- Idea: Combines neural language processing with explicit rules, logic, knowledge graphs, or verifiers.
Speculative category (future-hypothetical-model)
- Period: 2026โ2500
- Evidence status: speculative
- Idea: Potential post-2025 systems through 2500 cannot be catalogued as known facts; record proposals separately with evidence and uncertainty.
Important distinctions
Some entries are neural architectures (RNN, LSTM, Transformer, state-space models). Others are system patterns (retrieval-augmented generation, agentic systems), training/deployment methods (distillation, quantization, federated learning), or application specializations (code, speech, vision-language). They are not mutually exclusive: one deployed model may be a decoder-only Transformer, quantized, distilled, retrieval-augmented, and used in an agentic system.
Suggested SLeeLa model-card fields
Stable identifier; architecture and system-pattern tags; training objective; tokenizer; context length; positional strategy; layer count; hidden width; attention and expert dimensions; modalities; data provenance, license and cutoff; weight format and precision; evaluation results and limitations; safety and deployment constraints; evidence status.
A model configuration is not a trained model: weights, tokenizer, model card, checksums, and compatible inference implementation are separate artifacts. Do not infer capability from parameter count alone.