Documentation Index
Fetch the complete documentation index at: https://mintlify.com/itsubaki/gpt/llms.txt
Use this file to discover all available pages before exploring further.
itsubaki/gpt is a ground-up implementation of a GPT-style language model written entirely in Go. It covers the full pipeline — from training a BPE tokenizer on raw text, to pre-training a transformer model, to supervised fine-tuning for instruction following — with no Python or external ML frameworks required.
Introduction
Learn what itsubaki/gpt is, how it works, and when to use it.
Quickstart
Download pretrained weights and run text generation in minutes.
Architecture
Explore the transformer architecture: blocks, attention, RoPE, and norms.
Training Guide
Train a BPE tokenizer, pre-train a model, and fine-tune with instructions.
What’s Included
The library is organized into focused packages, each implementing a distinct part of the pipeline:BPE Tokenizer
Train and use a Byte Pair Encoding tokenizer from raw text corpora.
GPT Model
Construct, train, save, and reload a GPT model with configurable depth.
Text Generation
Generate text with temperature sampling or stream tokens via a Go channel.
Transformer Layers
Reusable layers: MultiHeadAttention, RMSNorm, SwiGLU, Linear, Embeddings.
Getting Started
This project is a companion to the O’Reilly book ゼロから作るDeep Learning ❻. It is designed for learning and experimentation, not production deployment.