Documentation Index
Fetch the complete documentation index at: https://mintlify.com/MilesONerd/neurenix/llms.txt
Use this file to discover all available pages before exploring further.
Overview
The utils module provides various utility functions for common tasks in deep learning, including seeding, data conversion, model management, and learning rate scheduling.
Random Seed Management
seed_everything
def seed_everything(seed: int) -> None
Data Conversion
to_numpy
def to_numpy(tensor: Union[Tensor, np.ndarray]) -> np.ndarray
tensor
Union[Tensor, np.ndarray]
required
Tensor or numpy array to convert.
to_tensor
def to_tensor(
data: Union[Tensor, np.ndarray, List, Tuple],
device: Optional[Union[str, Device]] = None
) -> Tensor
data
Union[Tensor, np.ndarray, List, Tuple]
required
Data to convert.
device
Optional[Union[str, Device]]
Device to store the tensor on.
one_hot
def one_hot(
indices: Union[Tensor, np.ndarray, List[int]],
num_classes: int
) -> Tensor
indices
Union[Tensor, np.ndarray, List[int]]
required
Class indices.
Model Management
get_module_device
def get_module_device(module: Module) -> Optional[Device]
Device of the module or None if the module has no parameters.
move_module_to_device
def move_module_to_device(module: Module, device: Union[str, Device]) -> None
device
Union[str, Device]
required
Device to move the module to.
count_parameters
def count_parameters(module: Module) -> int
Module to count parameters for.
Number of trainable parameters.
model_summary
def model_summary(module: Module) -> str
Optimizer Utilities
get_learning_rate
def get_learning_rate(optimizer: Optimizer) -> float
Optimizer to get learning rate from.
set_learning_rate
def set_learning_rate(optimizer: Optimizer, lr: float) -> None
Optimizer to set learning rate for.
Learning Rate Schedulers
StepLR
class StepLR:
def __init__(
self,
optimizer: Optimizer,
step_size: int,
gamma: float = 0.1,
last_epoch: int = -1
)
Optimizer to schedule learning rate for.
Period of learning rate decay.
Multiplicative factor of learning rate decay.
Methods
def step(self, epoch: Optional[int] = None) -> None
ExponentialLR
class ExponentialLR:
def __init__(
self,
optimizer: Optimizer,
gamma: float = 0.1,
last_epoch: int = -1
)
ReduceLROnPlateau
class ReduceLROnPlateau:
def __init__(
self,
optimizer: Optimizer,
mode: str = "min",
factor: float = 0.1,
patience: int = 10,
threshold: float = 1e-4,
threshold_mode: str = "rel",
cooldown: int = 0,
min_lr: float = 0.0,
eps: float = 1e-8,
verbose: bool = False,
)
One of ‘min’ or ‘max’. In ‘min’ mode, lr will be reduced when the quantity monitored has stopped decreasing.
Factor by which the learning rate will be reduced.
Number of epochs with no improvement after which learning rate will be reduced.
Methods
def step(self, metrics: float) -> None
Decorators
timeit
def timeit(func: Callable) -> Callable
Example Usage
import neurenix as nx
from neurenix.utils import (
seed_everything,
to_numpy,
to_tensor,
one_hot,
count_parameters,
model_summary,
StepLR,
ReduceLROnPlateau,
timeit
)
# Set random seed for reproducibility
seed_everything(42)
# Data conversion
np_array = np.array([1, 2, 3, 4, 5])
tensor = to_tensor(np_array, device="cuda:0")
back_to_numpy = to_numpy(tensor)
# One-hot encoding
class_indices = [0, 1, 2, 1, 0]
num_classes = 3
one_hot_encoded = one_hot(class_indices, num_classes)
print(one_hot_encoded.shape) # (5, 3)
# Model utilities
model = YourModel()
num_params = count_parameters(model)
print(f"Model has {num_params:,} parameters")
summary = model_summary(model)
print(summary)
# Get model device
device = get_module_device(model)
print(f"Model is on: {device}")
# Learning rate scheduling
optimizer = nx.optim.Adam(model.parameters(), lr=0.001)
# Step scheduler
scheduler = StepLR(optimizer, step_size=30, gamma=0.1)
for epoch in range(100):
train(model, optimizer)
scheduler.step()
print(f"Epoch {epoch}, LR: {get_learning_rate(optimizer)}")
# Reduce on plateau scheduler
scheduler = ReduceLROnPlateau(
optimizer,
mode='min',
factor=0.5,
patience=10,
verbose=True
)
for epoch in range(100):
train_loss = train(model, optimizer)
val_loss = validate(model)
scheduler.step(val_loss)
# Timing decorator
@timeit
def expensive_operation():
# Your code here
result = model(large_input)
return result
result = expensive_operation()
# Output: expensive_operation took 0.1234 seconds to execute
# Manual LR adjustment
set_learning_rate(optimizer, 0.0001)
print(f"New LR: {get_learning_rate(optimizer)}")
# Exponential decay
exp_scheduler = ExponentialLR(optimizer, gamma=0.95)
for epoch in range(50):
train(model, optimizer)
exp_scheduler.step()
Best Practices
Always seed: Call seed_everything() at the start of your script for reproducible results.
Monitor parameters: Use count_parameters() to understand your model size and memory requirements.
Learning rate scheduling: Start with ReduceLROnPlateau for adaptive learning rate adjustment based on validation metrics.
Profile code: Use the @timeit decorator to identify performance bottlenecks.