neurenix eval - Neurenix

The eval command evaluates a trained Neurenix model using specified metrics and test data.

Usage

neurenix eval --model <model_file> --data <data_path> [options]

Options

Option	Type	Required	Default	Description
`--model`	string	Yes	-	Path to the model file (`.nrx` format)
`--data`	string	Yes	-	Path to evaluation data (file or directory)
`--metrics`	string	No	`accuracy,precision,recall,f1`	Comma-separated list of metrics
`--output`	string	No	`evaluation.json`	Output file for evaluation results
`--batch-size`	integer	No	`32`	Batch size for evaluation
`--device`	string	No	`auto`	Device to use (`cpu`, `cuda`, `auto`)

Available Metrics

The following metrics are supported:

Classification Metrics

accuracy - Overall classification accuracy
precision - Precision score
recall - Recall score
f1 - F1 score (harmonic mean of precision and recall)
auc - Area under the ROC curve
confusion_matrix - Confusion matrix

Regression Metrics

mse - Mean squared error
rmse - Root mean squared error
mae - Mean absolute error
r2 - R-squared score

Custom Metrics

You can also specify custom metrics defined in your Neurenix configuration.

Examples

Basic evaluation

neurenix eval --model models/model.nrx --data data/test.csv

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.945
precision: 0.932
recall: 0.951
f1: 0.941

Saving evaluation results to evaluation.json...

Specify custom metrics

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,f1,auc

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, f1, auc...

Evaluation Results:
accuracy: 0.945
f1: 0.941
auc: 0.978

Saving evaluation results to evaluation.json...

Evaluate on directory of data

neurenix eval \
  --model models/image_classifier.nrx \
  --data data/test/ \
  --batch-size 64

Loading model from models/image_classifier.nrx...
Loading evaluation data from data/test/...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.892
precision: 0.884
recall: 0.897
f1: 0.890

Saving evaluation results to evaluation.json...

Custom output file

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --output results/test_metrics.json

Loading model from models/model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: accuracy, precision, recall, f1...

Evaluation Results:
accuracy: 0.945
precision: 0.932
recall: 0.951
f1: 0.941

Saving evaluation results to results/test_metrics.json...

Force CPU evaluation

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --device cpu

Regression metrics

neurenix eval \
  --model models/regression_model.nrx \
  --data data/test.csv \
  --metrics mse,rmse,mae,r2

Loading model from models/regression_model.nrx...
Loading evaluation data from data/test.csv...
Evaluating model with metrics: mse, rmse, mae, r2...

Evaluation Results:
mse: 0.0234
rmse: 0.1530
mae: 0.1121
r2: 0.923

Saving evaluation results to evaluation.json...

Output Format

The evaluation results are saved as JSON:

{
  "accuracy": 0.945,
  "precision": 0.932,
  "recall": 0.951,
  "f1": 0.941,
  "model": "models/model.nrx",
  "data": "data/test.csv",
  "batch_size": 32,
  "device": "cuda:0",
  "timestamp": "2026-03-08T15:30:45"
}

Data Format

CSV Files

For CSV files, the last column is treated as the label:

feature1,feature2,feature3,label
0.5,1.2,0.8,0
0.3,0.9,1.1,1

Directory Structure

For image classification, organize data by class:

data/test/
├── class_0/
│   ├── img1.jpg
│   ├── img2.jpg
│   └── ...
├── class_1/
│   ├── img1.jpg
│   ├── img2.jpg
│   └── ...
└── class_2/
    ├── img1.jpg
    └── ...

Error Handling

Model not found

neurenix eval --model nonexistent.nrx --data data/test.csv

Error: Model file 'nonexistent.nrx' not found.

Data not found

neurenix eval --model models/model.nrx --data missing.csv

Error: Data 'missing.csv' not found.

Invalid metric

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,invalid_metric

Error evaluating model: Unknown metric 'invalid_metric'

Best Practices

1. Use multiple metrics

Evaluate with comprehensive metrics:

neurenix eval \
  --model models/model.nrx \
  --data data/test.csv \
  --metrics accuracy,precision,recall,f1,auc,confusion_matrix

2. Separate test data

Keep test data completely separate from training:

# Never train on test data
neurenix run train.py  # Uses data/train and data/val

# Evaluate only after training is complete
neurenix eval --model models/model.nrx --data data/test.csv

3. Save results with meaningful names

neurenix eval \
  --model models/model_v1.nrx \
  --data data/test.csv \
  --output results/model_v1_test_metrics.json

4. Batch size for large datasets

Use appropriate batch sizes:

# Large dataset - increase batch size
neurenix eval \
  --model models/model.nrx \
  --data data/large_test/ \
  --batch-size 128

# Memory constrained - reduce batch size
neurenix eval \
  --model models/large_model.nrx \
  --data data/test.csv \
  --batch-size 8

5. Compare multiple models

# Evaluate baseline
neurenix eval \
  --model models/baseline.nrx \
  --data data/test.csv \
  --output results/baseline.json

# Evaluate improved model
neurenix eval \
  --model models/improved.nrx \
  --data data/test.csv \
  --output results/improved.json

# Compare results
diff results/baseline.json results/improved.json

Integration with Other Commands

After training

# Train model
neurenix run train.py

# Evaluate on test set
neurenix eval --model models/model.nrx --data data/test.csv

# Export if results are good
neurenix export --model models/model.nrx --format onnx

Before deployment

# Final evaluation
neurenix eval \
  --model models/production.nrx \
  --data data/test.csv \
  --output results/production_metrics.json

# If metrics are acceptable, serve
neurenix serve --model models/production.nrx --port 8000

​Usage

​Options

​Available Metrics

​Classification Metrics

​Regression Metrics

​Custom Metrics

​Examples

​Basic evaluation

​Specify custom metrics

​Evaluate on directory of data

​Custom output file

​Force CPU evaluation

​Regression metrics

​Output Format

​Data Format

​CSV Files

​Directory Structure

​Error Handling

​Model not found

​Data not found

​Invalid metric

​Best Practices

​1. Use multiple metrics

​2. Separate test data

​3. Save results with meaningful names

​4. Batch size for large datasets

​5. Compare multiple models

​Integration with Other Commands

​After training

​Before deployment

​See Also