Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/MilesONerd/neurenix/llms.txt

Use this file to discover all available pages before exploring further.

The eval command evaluates a trained Neurenix model using specified metrics and test data.

​
Usage

neurenix eval --model <model_file> --data <data_path> [options]

​
Options

OptionTypeRequiredDefaultDescription
--modelstringYes-Path to the model file (.nrx format)
--datastringYes-Path to evaluation data (file or directory)
--metricsstringNoaccuracy,precision,recall,f1Comma-separated list of metrics
--outputstringNoevaluation.jsonOutput file for evaluation results
--batch-sizeintegerNo32Batch size for evaluation
--devicestringNoautoDevice to use (cpu, cuda, auto)

​
Available Metrics

The following metrics are supported:

​
Classification Metrics

  • accuracy - Overall classification accuracy
  • precision - Precision score
  • recall - Recall score
  • f1 - F1 score (harmonic mean of precision and recall)
  • auc - Area under the ROC curve
  • confusion_matrix - Confusion matrix

​
Regression Metrics

  • mse - Mean squared error
  • rmse - Root mean squared error
  • mae - Mean absolute error
  • r2 - R-squared score

​
Custom Metrics

You can also specify custom metrics defined in your Neurenix configuration.

​
Examples

​
Basic evaluation

neurenix eval --model models/model.nrx --data data/test.csv

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.945
precision: 0.932
recall: 0.951
f1: 0.941

Saving evaluation results to evaluation.json...

​
Specify custom metrics

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,f1,auc

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, f1, auc...

Evaluation Results:
accuracy: 0.945
f1: 0.941
auc: 0.978

Saving evaluation results to evaluation.json...

​
Evaluate on directory of data

neurenix eval \
  --model models/image_classifier.nrx \
  --data data/test/ \
  --batch-size 64

Loading model from models/image_classifier.nrx...
Loading evaluation data from data/test/...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.892
precision: 0.884
recall: 0.897
f1: 0.890

Saving evaluation results to evaluation.json...

​
Custom output file

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --output results/test_metrics.json

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.945
precision: 0.932
recall: 0.951
f1: 0.941

Saving evaluation results to results/test_metrics.json...

​
Force CPU evaluation

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --device cpu

​
Regression metrics

neurenix eval \
  --model models/regression_model.nrx \
  --data data/test.csv \
  --metrics mse,rmse,mae,r2

Loading model from models/regression_model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: mse, rmse, mae, r2...

Evaluation Results:
mse: 0.0234
rmse: 0.1530
mae: 0.1121
r2: 0.923

Saving evaluation results to evaluation.json...

​
Output Format

The evaluation results are saved as JSON: 
{
  "accuracy": 0.945,
  "precision": 0.932,
  "recall": 0.951,
  "f1": 0.941,
  "model": "models/model.nrx",
  "data": "data/test.csv",
  "batch_size": 32,
  "device": "cuda:0",
  "timestamp": "2026-03-08T15:30:45"
}

​
Data Format

​
CSV Files

For CSV files, the last column is treated as the label: 
feature1,feature2,feature3,label
0.5,1.2,0.8,0
0.3,0.9,1.1,1

​
Directory Structure

For image classification, organize data by class: 
data/test/
├── class_0/
│   ├── img1.jpg
│   ├── img2.jpg
│   └── ...
├── class_1/
│   ├── img1.jpg
│   ├── img2.jpg
│   └── ...
└── class_2/
    ├── img1.jpg
    └── ...

​
Error Handling

​
Model not found

neurenix eval --model nonexistent.nrx --data data/test.csv

Error: Model file 'nonexistent.nrx' not found.

​
Data not found

neurenix eval --model models/model.nrx --data missing.csv

Error: Data 'missing.csv' not found.

​
Invalid metric

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,invalid_metric

Error evaluating model: Unknown metric 'invalid_metric'

​
Best Practices

​
1. Use multiple metrics

Evaluate with comprehensive metrics: 
neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,precision,recall,f1,auc,confusion_matrix

​
2. Separate test data

Keep test data completely separate from training: 
# Never train on test data
neurenix run train.py  # Uses data/train and data/val

# Evaluate only after training is complete
neurenix eval --model models/model.nrx --data data/test.csv

​
3. Save results with meaningful names

neurenix eval \
  --model models/model_v1.nrx \
  --data data/test.csv \
  --output results/model_v1_test_metrics.json

​
4. Batch size for large datasets

Use appropriate batch sizes: 
# Large dataset - increase batch size
neurenix eval \
  --model models/model.nrx \
  --data data/large_test/ \
  --batch-size 128

# Memory constrained - reduce batch size
neurenix eval \
  --model models/large_model.nrx \
  --data data/test.csv \
  --batch-size 8

​
5. Compare multiple models

# Evaluate baseline
neurenix eval \
  --model models/baseline.nrx \
  --data data/test.csv \
  --output results/baseline.json

# Evaluate improved model
neurenix eval \
  --model models/improved.nrx \
  --data data/test.csv \
  --output results/improved.json

# Compare results
diff results/baseline.json results/improved.json

​
Integration with Other Commands

​
After training

# Train model
neurenix run train.py

# Evaluate on test set
neurenix eval --model models/model.nrx --data data/test.csv

# Export if results are good
neurenix export --model models/model.nrx --format onnx

​
Before deployment

# Final evaluation
neurenix eval \
  --model models/production.nrx \
  --data data/test.csv \
  --output results/production_metrics.json

# If metrics are acceptable, serve
neurenix serve --model models/production.nrx --port 8000

​
See Also

Build docs developers (and LLMs) love

Get started for freeTalk to us