Time-MoE

​

Overview

​

Configuration

from samay.utils import load_args

arg_path = "config/timemoe.json"
args = load_args(arg_path)

{
    "repo": "Maple728/TimeMoE-50M",
    "context_len": 512,
    "horizon_len": 96
}

​

Loading the Model

from samay.model import TimeMoEModel
from samay.utils import load_args

arg_path = "config/timemoe.json"
args = load_args(arg_path)
model = TimeMoEModel(**args)

​

Loading Dataset

from samay.dataset import TimeMoEDataset

train_dataset = TimeMoEDataset(
    name="ett",
    datetime_col="date",
    path="./data/ETTh1.csv",
    mode="train",
    batch_size=32,
    context_len=512,
    horizon_len=96,
    task_name="finetune"
)

val_dataset = TimeMoEDataset(
    name="ett",
    datetime_col="date",
    path="./data/ETTh1.csv",
    mode="test",
    batch_size=128,
    context_len=512,
    horizon_len=96
)

​

Zero-Shot Forecasting

metrics = model.evaluate(val_dataset, metric_only=True)
print(metrics)
# {'mse': ..., 'mae': ..., 'mase': ..., 'mape': ..., ...}

​

Fine-tuning

# Note: Fine-tuning implementation is being improved
model.finetune(train_dataset)

​

Visualization

model.plot(val_dataset)

​

Mixture of Experts Architecture

1. Sparse Gating: Routes inputs to a subset of experts
2. Specialized Experts: Each expert learns different patterns
3. Efficient Scaling: Increases capacity without proportional compute increase
4. Load Balancing: Ensures even distribution across experts

​

Key Features

• Scalable: Efficiently scales to billion-parameter models
• Sparse Activation: Only a subset of parameters active per input
• Specialized Learning: Different experts capture different patterns
• Efficient Inference: Despite large size, inference is efficient

​

Model Variants

​

When to Use Time-MoE

• When you need state-of-the-art accuracy
• For diverse time series patterns in your data
• When you have computational resources for larger models
• For complex forecasting tasks requiring specialized expertise

​

Example Notebook

• example/timemoe.ipynb