Frankenstein Transformer
Configuration-driven transformer experimentation toolkit with 35+ sequence mixer architectures and 23 optimizer families, strict schema-based configuration, quantized BitNet deployment, and SBERT sentence-embedding training.
Full hosted documentation is at https://frankenstein-transformer.readthedocs.io.
Overview:
Specifications:
- Specifications
- Architecture
- System Architecture Specification
- Attention Mixers
- Sequence Mixer Families Specification
- Activation Functions
- Activation Functions Specification
- CLI Reference
- CLI Command Reference
- Entrypoint
- Subcommand Overview
- Device Choices
train— Main Trainingdeploy— Deployment Artifact Creationquantize— Quantized Exportinfer— Model Inferencesbert-train— SBERT Trainingsbert-infer— SBERT Inferenceweb-server— Streamlit Configuration Buildertransformers-export— HuggingFace Export- GPU Thermal Guard Flags
- Deployment
- Deployment and Quantization Specification
- mHC: Manifold-Constrained Hyper-Connections
- mHC: Manifold-Constrained Hyper-Connections
- Optimizers
- Optimizer Families Specification
- SBERT Workflows
- SBERT Training and Inference Specification
- Schema Reference
- Schema Field Reference
- Training Safety
- Training Safety and Stability Specification
- Vision Transformer (frankenstein_vit)
API Reference:
Bibliography:
- Bibliography
- Advanced Sequence Modeling Architectures
- Advanced Sequence Modeling Architectures: A Comprehensive Theoretical and Empirical Analysis of Transformer Blocks and Emerging Paradigms
- Sparse Attention Architectures
- Review of Sparse Attention Blocks in Transformers
- Gated Attention Architectures
- Gated Attention Blocks in Transformers: A Literature Review
- Optimizer Families
- Advanced Optimization Algorithms in Transformer Architectures: A Comprehensive Analysis
- Attention Types Bibliography
- Optimizers Bibliography
- Other Bibliography
Technical Reports: