Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/itsubaki/gpt/llms.txt

Use this file to discover all available pages before exploring further.

This quickstart walks through running the pretrained model for code generation and instruction-following chat, with no training required. You will download the serialized weights from the gob branch, run a code-completion prompt against the base model, and send an instruction to the SFT chat model — all in under five minutes.
1

Prerequisites

You need Go 1.26 or later and git installed. Clone the repository and enter the project directory:
git clone https://github.com/itsubaki/gpt
cd gpt
No additional tools or Python environment are required. All dependencies are pure Go and are fetched automatically by go run.
2

Download Pretrained Data

The make testdata target downloads four pre-built binary files from the gob branch of the repository into the testdata/ directory:
make testdata
Under the hood this runs:
curl -fs -o testdata/merge_rules.gob   https://raw.githubusercontent.com/itsubaki/gpt/refs/heads/gob/testdata/merge_rules.gob
curl -fs -o testdata/tiny_codes.bin    https://raw.githubusercontent.com/itsubaki/gpt/refs/heads/gob/testdata/tiny_codes.bin
curl -fs -o testdata/model_gpt.gob     https://raw.githubusercontent.com/itsubaki/gpt/refs/heads/gob/testdata/model_gpt.gob
curl -fs -o testdata/model_gpt_sft.gob https://raw.githubusercontent.com/itsubaki/gpt/refs/heads/gob/testdata/model_gpt_sft.gob
FileContents
merge_rules.gobSerialized BPE merge rules for the 1 000-token vocabulary
tiny_codes.binPre-tokenized training corpus (token IDs as gob-encoded []int)
model_gpt.gobPre-trained base model weights
model_gpt_sft.gobSupervised fine-tuned (SFT/chat) model weights
3

Generate Code

Run the generate command with a Python function signature as the prompt:
go run ./cmd/generate/main.go --prompt 'def add(a, b):' --temperature 0.3
Expected output:
model parameters:
 VocabSize    : 1000
 MaxContextLen: 256
 EmbedDim     : 192
 NumOfHeads   : 6
 NumOfBlocks  : 6
------------------------------
300,890,40,97,44,358,281,259,312,358,390,365,58,272,301,428,97,41,259,301,273,347,358,271,307,40,97,44,358,41,10,999,
------------------------------
def add(a, b):
    if b == 0:
        return (a)
    return a + b

print(a, b)
The comma-separated integers are the raw token IDs emitted by the streaming channel. The decoded text appears below the separator.
Pass --temperature 0.0 to use greedy (argmax) decoding for fully deterministic output. This is useful for reproducible tests and benchmarks.
4

Chat with the Fine-Tuned Model

The SFT model understands natural-language instructions in Alpaca format. Use the chat command to send it a prompt:
go run ./cmd/chat/main.go --prompt 'Write is_prime function'
Expected output:
model parameters:
 VocabSize    : 1000
 MaxContextLen: 256
 EmbedDim     : 192
 NumOfHeads   : 6
 NumOfBlocks  : 6
------------------------------
35,35,35,955,435,117,387,58,10,87,903,890,618,271,35,35,35,608,101,966,58,10,300,890,40,97,44,358,281,259,301,273,347,358,999,
------------------------------
### Instruction:
Write is_prime function

### Response:
def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            return False
    return True
The chat command internally wraps your prompt in the ### Instruction: / ### Response: Alpaca template before feeding it to the model.

​
Running Examples

The example Makefile target runs a curated set of generate and chat prompts back-to-back to verify your setup: 
make example
This executes the following commands: 
go run ./cmd/generate/main.go --model-path testdata/model_gpt.gob --temperature 0.3 --prompt 'def add(a, b):'
go run ./cmd/generate/main.go --model-path testdata/model_gpt.gob --temperature 0.3 --prompt 'def factorial(n):'
go run ./cmd/generate/main.go --model-path testdata/model_gpt.gob --temperature 0.3 --prompt 'def fibonacci(n):'
go run ./cmd/generate/main.go --model-path testdata/model_gpt.gob --temperature 0.3 --prompt 'def is_prime(n):'
go run ./cmd/generate/main.go --model-path testdata/model_gpt.gob --prompt 'def'
go run ./cmd/chat/main.go --prompt 'Write is_prime function'
go run ./cmd/chat/main.go --prompt 'Hi, who are you?'
go run ./cmd/chat/main.go --prompt '3+7'
The first four prompts exercise code completion at low temperature; the fifth uses the default temperature of 1.0 for more varied output. The final three prompts demonstrate the SFT chat model responding to an instruction, a conversational question, and a simple arithmetic expression.

​
Using as a Library

You can embed the model directly in your own Go program. Import the model and tokenizer packages and call the high-level helpers: 
import (
    "fmt"

    "github.com/itsubaki/gpt/model"
    "github.com/itsubaki/gpt/tokenizer"
)

func main() {
    // Load the BPE tokenizer from serialized merge rules
    tknizer, err := tokenizer.NewBPETokenizerFrom("testdata/merge_rules.gob")
    if err != nil {
        panic(err)
    }

    // Load the pre-trained model with KV cache enabled
    m, err := model.NewGPTFrom("testdata/model_gpt.gob", true)
    if err != nil {
        panic(err)
    }

    // Generate up to 64 new tokens with temperature 0.3
    text := model.GenerateText(m, m.MaxContextLen, tknizer, "def add(a, b):", 64, 0.3)
    fmt.Println(text)
}
For streaming output, use model.GenerateChan instead. It returns a <-chan int that emits one token ID at a time so you can print or process tokens as they are produced: 
ch := model.GenerateChan(m, m.MaxContextLen, tknizer, "def add(a, b):", 64, 0.3)
for id := range ch {
    fmt.Print(tknizer.Decode([]int{id}))
}
fmt.Println()
The second argument to model.NewGPTFrom controls the KV cache. Pass true during inference for significantly faster generation — the cache avoids recomputing key and value projections for tokens that have already been processed.

Build docs developers (and LLMs) love

Get started for freeTalk to us