Constraints files allow you to guide ML Experiment Autopilot’s decisions using natural language. Gemini interprets these preferences when designing experiments, selecting models, and choosing preprocessing strategies.
# Experiment Constraints## Metrics- Primary metric: RMSE## Models- Prefer tree-based models- Prefer boosting methods## Preprocessing- Log-transform the target variable- Use median imputation for missing values## Termination- Stop if no improvement for 3 iterations
Save constraints as .md files and reference with --constraints path/to/file.md.
## Models- Prefer tree-based models (RandomForest, XGBoost, LightGBM)- Avoid linear models (dataset has non-linear patterns)- Start with ensemble methods
## Models- Only use scikit-learn models (no XGBoost/LightGBM)- Prefer interpretable models (LogisticRegression, DecisionTree)- Fast training required (limit to n_estimators < 100)
Available models (from src/execution/code_generator.py):
## Preprocessing- Log-transform the target variable (right-skewed distribution)- Use median imputation for missing values (robust to outliers)- Standard scaling for numeric features- One-hot encoding for categorical features
## Preprocessing- Do NOT transform the target variable- Drop rows with missing values (only 2% missing)- Min-max scaling to [0, 1] range- Ordinal encoding for categorical features (natural ordering)
Preprocessing options (defined in src/orchestration/state.py:48-54):
High-level guidance for the experimental approach:
## Strategy- Explore diverse model families in first 5 iterations- Then exploit best-performing approach- Prioritize generalization over training performance
## Strategy- Focus on interpretability (business requirement)- Avoid black-box models (XGBoost, neural networks)- Prefer simpler models with similar performance
# California Housing Price Prediction Constraints## Metrics- Primary metric: RMSE- Also track MAE for interpretability## Models- Prefer tree-based models (capture non-linear relationships)- Try boosting methods (XGBoost, LightGBM, GradientBoosting)- Avoid linear models (preliminary EDA shows non-linearity)## Preprocessing- Log-transform the target variable MedHouseVal (right-skewed distribution)- Use median imputation for missing values in total_bedrooms (207 missing)- Standard scaling for all numeric features- No categorical features in this dataset## Hyperparameters- Limit tree depth to avoid overfitting (max_depth <= 10)- Use conservative learning rates (0.01 - 0.1)## Termination- Stop if no improvement for 3 iterations- Target RMSE: 0.20 or better- Maximum 15 iterations## Notes- Dataset has 20,640 samples, sufficient for complex models- Focus on generalization (avoid overfitting to training set)
# Bank Marketing Classification Constraints## Metrics- Primary metric: F1 score (imbalanced dataset: 11.7% positive class)- Also track precision and recall separately- Accuracy is misleading (do not optimize for accuracy alone)## Models- Prefer models that handle imbalance well: - LogisticRegression with class_weight='balanced' - RandomForestClassifier with balanced class weights - GradientBoostingClassifier- Avoid naive models that predict majority class## Preprocessing- Do NOT transform the target (binary classification)- Mean imputation for numeric features (minimal missing values)- One-hot encoding for categorical features (age_group, job, marital, etc.)- Standard scaling for numeric features## Class Imbalance- Address 8:1 class imbalance- Consider SMOTE or class weights- Evaluate on both classes, not just majority## Hyperparameters- Use class_weight='balanced' where supported- Avoid high complexity (prevents overfitting to minority class noise)## Termination- Stop if F1 > 0.65 (good performance on imbalanced data)- Stop if no improvement for 4 iterations- Maximum 12 iterations## Notes- Precision important (avoid false positives in marketing)- Recall important (capture potential customers)- Balance via F1 score optimization
- Prefer tree-based models for non-linear relationships- Try XGBoost and LightGBM with tuned learning rates
Too Vague:
- Use good models- Make it work well
Too Rigid:
- Only use XGBRegressor with max_depth=5, learning_rate=0.05, n_estimators=100- Never deviate from these exact parameters
Provide rationale for preferences. Gemini considers “why” when deciding whether to follow or deviate.
Include Domain Knowledge
## Domain Knowledge- Target variable (house prices) is right-skewed → log transformation likely helps- Features have different scales (income vs. age) → scaling important- Missing values in total_bedrooms are not random → median imputation reasonable- Non-linear price relationships observed in EDA → prefer non-linear models