Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/itsubaki/gpt/llms.txt

Use this file to discover all available pages before exploring further.

Supervised fine-tuning (SFT) adapts a pre-trained GPT model to follow instructions by continuing training on (instruction, response) pairs formatted in the Alpaca style. The prompt portion of each sequence is masked from the loss with label index -100, so only the response tokens contribute to gradient updates. This focuses learning on generating correct completions rather than re-learning the prompt structure.

​
Alpaca format

Each training example is assembled from an Alpaca struct containing instruction and response fields: 
type Alpaca struct {
    Instruction string `json:"instruction"`
    Response    string `json:"response"`
}
The AlpacaFormat() function builds the prompt string: 
func AlpacaFormat(message string) string {
    return fmt.Sprintf("### Instruction:\n%s\n\n### Response:\n", message)
}
The full input to the model concatenates the prompt with the response followed by the end-of-text token: 
### Instruction:
{instruction}

### Response:
{response}<|endoftext|>

​
Label masking

The NewAlpacaDataset() function encodes each example and fills the prompt positions with -100: 
promptIDs := tokenizer.Encode(prompt)
responseIDs := tokenizer.Encode(response)

ids := append(promptIDs, responseIDs...)
labels := make([]int, len(promptIDs))
for i := range labels {
    labels[i] = -100   // prompt tokens are ignored in the loss
}
labels = append(labels, responseIDs...)
The cross-entropy loss in cmd/sft/main.go uses the default ignore index of -100: 
loss := F.CrossEntropy(
    F.Reshape(-1, logits.Size(-1))(logits), // (B, C, V) -> (B*C, V)
    F.Reshape(-1)(y),                       // (B, C) -> (B*C)
    // ignore index -100
)
Sequences shorter than contextLen are right-padded with 0 (input) and -100 (labels). Sequences longer than contextLen are truncated.

​
CLI flags

FlagDefaultDescription
-context-len256Maximum context length
-max-learning-rate3e-4Peak learning rate for AdamW
-beta10.9AdamW first moment decay
-beta20.999AdamW second moment decay
-weight-decay0.01L2 weight decay for AdamW
-clip1.0Gradient clipping norm threshold
-max-iters500Total fine-tuning iterations
-batch-size32Number of sequences per batch
-merge-rules-pathtestdata/merge_rules.gobPath to BPE merge rules for the tokenizer
-model-pathtestdata/model_gpt.gobPath to the pre-trained base model
-alpaca-pathtestdata/tiny_codes_sft.jsonPath to Alpaca-format instruction JSON
-sft-model-pathtestdata/model_gpt_sft.gobOutput path for the fine-tuned model
-min-loss1.0Saves model_gpt_sft.gob.min when beaten
-pproffalseEnable CPU profiling to cpu_sft.prof

​
Running SFT

% make sft
go run ./cmd/sft/main.go
SFT 100%|██████████████████████████████| 500/500
The progress bar reports loss and perplexity at each step: 
SFT           100%|██████████████████████████████| 500/500 [0.4m<0.0s, 21.3 it/s] loss=0.8741(ppl=2.3964)
Loss is also written to loss_sft.csv every iteration for plotting.

​
SFT vs pre-training

Pre-TrainingFine-Tuning
Iterations20,000500
Datasettiny_codes.bin (token IDs)tiny_codes_sft.json (instruction pairs)
Loss targetAll tokensResponse tokens only
Model weightsRandomly initialisedLoaded from model_gpt.gob
Outputmodel_gpt.gobmodel_gpt_sft.gob
SFT runs for only 500 iterations because the model already has language understanding from pre-training — it only needs to learn the instruction-following pattern, not the underlying language. Training for too many iterations risks overfitting the small SFT dataset.

​
Example output

After SFT, the model responds to instructions in the structured format: 
### Instruction:
Write is_prime function

### Response:
def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            return False
    return True
Additional examples from the trained model: 
### Instruction:
Write add function

### Response:
def add(a, b):
    return a + b

### Instruction:
Hi, who are you?

### Response:
I'm an AI assistant. What do you need help with?

### Instruction:
3+9

### Response:
12
The pre-trained fine-tuned model (model_gpt_sft.gob) is available for download. Run make testdata to fetch all four pre-built artifacts — merge rules, token binary, base model, and SFT model — without running any training yourself.
Increase -max-iters beyond 500 if the model produces inconsistent or incomplete responses. For small datasets like tiny_codes_sft.json, values in the range 1000–2000 can improve instruction-following quality, though you should monitor loss_sft.csv for signs of overfitting.

Build docs developers (and LLMs) love

Get started for freeTalk to us