Specifications

• Architecture
• System Architecture Specification
  • System Design Overview
  • Model Classes
  • Training Modes
  • Looped Depth Formula
  • Layer Pattern Dispatcher
  • Mixture-of-Depths (MoD) Routing
  • Engram Conditional Memory
  • BitNet Path
  • Factorized Embeddings
  • Embedding Convolution
  • Normalization Variants
  • Positional Encodings
  • MoE FFN Routing
• Attention Mixers
• Sequence Mixer Families Specification
  • Taxonomy Overview
  • Training-Free Policy
  • Dense Attention Baselines (2)
  • Grouped-Query Attention (1)
  • Recurrent and Retentive Architectures (5)
  • Sparse Attention Patterns (7)
  • Gated Attention Mechanisms (6)
  • Latent Attention Mechanisms (2)
  • Comprehensive Comparison Table
• Activation Functions
• Activation Functions Specification
  • Overview
  • Family Decision Tree
  • Selection Contract
  • ffn_activation_config Keys
  • Rational Activation Function (RAF)
  • GLU Variants
  • Key Files
  • References
• CLI Reference
• CLI Command Reference
  • Entrypoint
  • Subcommand Overview
  • Device Choices
  • train — Main Training
  • deploy — Deployment Artifact Creation
  • quantize — Quantized Export
  • infer — Model Inference
  • sbert-train — SBERT Training
  • sbert-infer — SBERT Inference
  • web-server — Streamlit Configuration Builder
  • transformers-export — HuggingFace Export
  • GPU Thermal Guard Flags
• Deployment
• Deployment and Quantization Specification
  • Deploy Pipeline
  • Quantization Methods
  • Size Estimates
  • Deployment Artifact Contents
  • Inference Modes
  • Quantization CLI
  • Validation
  • HuggingFace Transformers Export
  • BitNet.cpp / GGUF Export (best-effort)
• mHC: Manifold-Constrained Hyper-Connections
• mHC: Manifold-Constrained Hyper-Connections
  • Overview
  • Mathematical formulation
  • Implementation (Frankenstein)
  • Config reference
  • Constraints
• Optimizers
• Optimizer Families Specification
  • Optimizer Routing Framework
  • Prefixed Key Contract
  • Complete Optimizer Inventory
  • Optimizer Family Groupings
  • Key Optimizer Equations
• SBERT Workflows
• SBERT Training and Inference Specification
  • Overview
  • Siamese Training
  • Pooling Modes
  • Dataset Types
  • SBERT Configuration Block
  • Inference Modes
  • SBERT Inference Mode Router (Pseudocode)
  • CLI Examples
• Schema Reference
• Schema Field Reference
  • Strict Validation Policy
  • Top-Level Sections
  • Model Fields
  • Training Fields
  • Optimizer Fields
  • Scheduler Types
  • GPU Metrics Backends
  • SBERT Dataset Types
  • Critical Gotchas
• Training Safety
• Training Safety and Stability Specification
  • Gradient Accumulation
  • Global Norm Clipping
  • Post-Clip Explosion Guard
  • NaN/Inf Retry Logic
  • Checkpointing Policy
  • Telemetry
  • GPU Thermal Guard
  • GaLore Controls
  • Scheduler Types
  • Training Step Algorithm (Full)
  • Stability Best Practices
• Vision Transformer (frankenstein_vit)