Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/itsubaki/gpt/llms.txt

Use this file to discover all available pages before exploring further.

itsubaki/gpt is a ground-up implementation of a GPT-style language model written entirely in Go. It covers the full pipeline — from training a BPE tokenizer on raw text, to pre-training a transformer model, to supervised fine-tuning for instruction following — with no Python or external ML frameworks required.

Introduction

Learn what itsubaki/gpt is, how it works, and when to use it.

Quickstart

Download pretrained weights and run text generation in minutes.

Architecture

Explore the transformer architecture: blocks, attention, RoPE, and norms.

Training Guide

Train a BPE tokenizer, pre-train a model, and fine-tune with instructions.

What’s Included

The library is organized into focused packages, each implementing a distinct part of the pipeline:

BPE Tokenizer

Train and use a Byte Pair Encoding tokenizer from raw text corpora.

GPT Model

Construct, train, save, and reload a GPT model with configurable depth.

Text Generation

Generate text with temperature sampling or stream tokens via a Go channel.

Transformer Layers

Reusable layers: MultiHeadAttention, RMSNorm, SwiGLU, Linear, Embeddings.

Getting Started

1

Install the module

Add itsubaki/gpt to your Go project:
go get github.com/itsubaki/gpt
2

Download pretrained artifacts

Fetch pretrained merge rules and model weights from the gob branch:
make testdata
3

Generate text

Run the generate command with a code prompt:
go run ./cmd/generate/main.go --prompt 'def add(a, b):' --temperature 0.3
4

Chat with the fine-tuned model

Ask the SFT model a coding question:
go run ./cmd/chat/main.go --prompt 'Write is_prime function'
This project is a companion to the O’Reilly book ゼロから作るDeep Learning ❻. It is designed for learning and experimentation, not production deployment.

Build docs developers (and LLMs) love