Representative architecture families and model-system patterns known through 2025.
This is not a list of 2,000–2,500 individual models. Future possibilities through 2500 are speculative, not established knowledge.
Statistical language model
2000s
Predicts tokens from local frequency context; useful baseline, limited long-range structure.
Probabilistic sequence model
pre-2010
Models hidden states and observed token sequences; historically important for speech and NLP.
Structured prediction
2000s
Labels sequences using contextual feature dependencies in classical NLP pipelines.
Neural language model
2000s
Predicts a token from a fixed context window using dense neural layers.
Recurrent neural network
2010s
Processes sequences with recurrent state; foundational neural sequence architecture.
Gated recurrent network
2010s
Uses input, forget, and output gates to retain information over longer sequences.
Gated recurrent network
2010s
A streamlined gated recurrent architecture with update and reset gates.
Encoder-decoder sequence model
2010s
Maps an input sequence to an output sequence, often with recurrent or transformer components.
Attention-based encoder-decoder
2010s
Allows decoder steps to focus on relevant encoded input positions.
Bidirectional encoder
2018 onward
Uses self-attention and feed-forward blocks for contextual representations; BERT-style family.
Autoregressive decoder
2018 onward
Uses causal self-attention to predict the next token; GPT-style family.
Text-to-text transformer
2019 onward
Uses an encoder and autoregressive decoder for translation, summarization, and general text tasks.
Conditional-compute model
2020s
Routes tokens or examples to a subset of expert networks to scale capacity with conditional computation.
Retrieval-grounded model system
2020s
Combines a language model with search or document retrieval; retrieved evidence is distinct from model weights.
Extended-context model
2020s
Uses attention variants, memory, or position strategies to process longer sequences.
State-space sequence model
2020s
Uses structured state-space dynamics or selective scans as an alternative to full attention in some settings.