Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/itsubaki/gpt/llms.txt

Use this file to discover all available pages before exploring further.

The cmd/chat command wraps the SFT (supervised fine-tuning) trained model with the Alpaca instruction format. Before running generation it automatically prepends ### Instruction:\n{prompt}\n\n### Response:\n to the user’s input, framing it as an instruction the model was trained to follow. The model then generates the response portion autoregressively until the end token is produced.

​
CLI usage

The cmd/chat command shares the same generation flags as cmd/generate but loads the SFT model by default via -sft-model-path.
FlagDefaultDescription
-merge-rules-pathtestdata/merge_rules.gobPath to the BPE merge rules file
-sft-model-pathtestdata/model_gpt_sft.gobPath to the SFT fine-tuned model file
-promptWrite loopInstruction to send to the model
-temperature1.0Sampling temperature
-max-new-tokens256Maximum number of new tokens to generate
-count1Number of generation runs to perform
Run the default chat command: 
make chat
# equivalent to:
go run ./cmd/chat/main.go

​
Example outputs

Writing a Python function — --prompt 'Write is_prime function': 
go run ./cmd/chat/main.go --prompt 'Write is_prime function'

### Instruction:
Write is_prime function

### Response:
def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            return False
    return True
Conversational prompt — --prompt 'Hi, who are you?': 
go run ./cmd/chat/main.go --prompt 'Hi, who are you?'

### Instruction:
Hi, who are you?

### Response:
I'm an AI assistant. What do you need help with?
Simple arithmetic — --prompt '3+9': 
go run ./cmd/chat/main.go --prompt '3+9'

### Instruction:
3+9

### Response:
12

​
Alpaca format

The chat command uses dataloader.AlpacaFormat() to wrap the user prompt before passing it to GenerateChan. This matches the format used during SFT training, so the model knows where the instruction ends and where to begin generating its response. 
// From dataloader/dataset_alpaca.go
func AlpacaFormat(message string) string {
    return fmt.Sprintf("### Instruction:\n%s\n\n### Response:\n", message)
}
Inside cmd/chat/main.go, the wrapped prompt is passed directly to generation: 
ch := model.GenerateChan(
    m,
    m.MaxContextLen,
    tknizer,
    dataloader.AlpacaFormat(prompt), // wraps the raw prompt
    maxNewTokens,
    temperature,
)
The model generates tokens starting from the ### Response: marker, so the decoded output includes the full Alpaca-formatted exchange.

​
Library usage

Use dataloader.AlpacaFormat together with model.GenerateText or model.GenerateChan to chat with the SFT model from your own Go program: 
import (
    "fmt"

    "github.com/itsubaki/gpt/dataloader"
    "github.com/itsubaki/gpt/model"
    "github.com/itsubaki/gpt/tokenizer"
)

tknizer, _ := tokenizer.NewBPETokenizerFrom("testdata/merge_rules.gob")
m, _ := model.NewGPTFrom("testdata/model_gpt_sft.gob", true)

prompt := dataloader.AlpacaFormat("Write add function")
text := model.GenerateText(m, m.MaxContextLen, tknizer, prompt, 128, 1.0)
fmt.Println(text)
This will print the full Alpaca-formatted exchange, including the ### Instruction: and ### Response: headers, followed by the generated Python code.
The chat command loads testdata/model_gpt_sft.gob — the supervised fine-tuned model. Use the generate command with testdata/model_gpt.gob if you want to run the base pre-trained model without instruction formatting.
The SFT model was trained on a small Python code dataset (tiny_codes_sft.json). It works best with Python coding prompts such as “Write a function that…” or “Implement …”. Responses to general knowledge or non-Python questions may be less reliable.

Build docs developers (and LLMs) love

Get started for freeTalk to us