Pipeline Artifacts: All Files Produced by ExoProfiler

ExoProfiler produces a set of versioned, reusable artifacts at each pipeline stage. These include clean data matrices, fitted sklearn transformers, PCA models, cluster assignments, and JSON contracts. All artifacts are written to data/processed/ and its subdirectories.

Directory Layout

data/processed/
├── exoplanets_preprocessed.csv        # Clean 731 × 12 feature matrix
├── exoplanets_selected_raw.csv        # Original values for retained rows
├── preprocessing_row_audit.csv        # Per-planet include/exclude decision
├── preprocessing_quality_summary.csv  # Quality metrics
├── preprocessing_scaler_comparison.csv
├── preprocessing_imputer_comparison.csv
├── preprocessing_pipeline.joblib      # Fitted sklearn pipeline
├── preprocessing_metadata.json        # Pipeline contract
├── clustered_exoplanets.csv           # Final labeled dataset
└── pca/
    ├── pca_scores.csv                 # 731 × 4 component scores
    ├── pca_explained_variance.csv     # Per-component variance
    ├── pca_loadings.csv               # Variable contributions
    ├── pca_model.joblib               # Fitted PCA model
    └── pca_metadata.json             # PCA contract

Core Data Artifacts

exoplanets_preprocessed.csv

Property	Value
Shape	731 rows × 13 columns (`pl_name` + 12 features)
Null guarantee	Zero nulls, zero infinities, zero duplicate identifiers
Column order	Matches `FEATURE_COLUMNS` exactly
Produced by	Notebook 03

This is the primary output of the preprocessing stage and the direct input consumed by notebook 04 (PCA). The 12 feature columns are log1p-transformed where applicable, RobustScaler-scaled, and KNN-imputed. The pl_name column is carried as an identifier only and is never used as a model variable.

exoplanets_selected_raw.csv

Property	Value
Shape	731 rows × 13 columns
Contents	Original (un-transformed, un-scaled) values for the 731 retained planets
Produced by	Notebook 03

This file preserves the physical-unit values before any transformation or scaling. It is useful for interpreting cluster results in their original astronomical units — for example, reading orbital periods in days or stellar masses in solar masses rather than the normalized values in exoplanets_preprocessed.csv.

clustered_exoplanets.csv

Property	Value
Shape	731 rows × 7 columns
Columns	`pl_name`, `PC1`, `PC2`, `PC3`, `PC4`, `Cluster_K2`, `Familia_Planeta`
Produced by	Notebook 05

Column details:

Column	Type	Values
`pl_name`	string	Unique planet identifier
`PC1`–`PC4`	float	PCA component scores from `pca_scores.csv`
`Cluster_K2`	integer	`0` = Standard, `1` = Exotic
`Familia_Planeta`	string	`'Familia 0 (Población Estándar)'` or `'Familia 1 (Población Exótica / Atípica)'`

This is the final, end-to-end labeled dataset joining PCA coordinates with K-Means assignments. Cluster 0 contains 667 planets (91.24%); Cluster 1 contains 64 planets (8.76%).

pca/pca_scores.csv

Property	Value
Shape	731 rows × 5 columns
Columns	`pl_name`, `PC1`, `PC2`, `PC3`, `PC4`
Produced by	Notebook 04

The four component scores represent the 90.35% of total variance retained by the 4-component PCA solution. These scores are the direct input to K-Means in notebook 05. The join between pca_scores.csv and downstream artifacts is always done by pl_name, not by row position.

Audit Artifacts

preprocessing_row_audit.csv

Per-planet record of the include/exclude decision with reason. Use this file to trace why specific planets were filtered out of the 910-planet raw dataset. Planets with fewer than 8 of 12 features observed are marked as excluded.

preprocessing_quality_summary.csv

Summary quality metrics for the preprocessed dataset. Documents null counts, infinity counts, and duplicate checks after each pipeline step.

preprocessing_scaler_comparison.csv

Descriptive statistics comparing StandardScaler and RobustScaler on the 12 features. Documents the statistical justification for choosing RobustScaler, which uses median and interquartile range rather than mean and standard deviation, making it more resistant to extreme values.

preprocessing_imputer_comparison.csv

Benchmark results from the KNN vs. median imputation test. The test used the 249 complete-case planets, hid 10% of values at random (seed 42), and measured reconstruction error in the transformed and scaled space.

Method	RMSE	MAE
Median	0.7168	0.5651
KNN (5 neighbors, distance weights)	0.4026	0.2422

pca/pca_explained_variance.csv

Two-column file containing the component name alongside its individual and cumulative variance ratios. Use this file to reproduce the scree plot or to verify how many components are needed for a different variance threshold.

Component	Individual Variance	Cumulative Variance
PC1	44.65%	44.65%
PC2	29.56%	74.21%
PC3	9.41%	83.62%
PC4	6.74%	90.35%

pca/pca_loadings.csv

Matrix of shape (12 variables × 4 components). Each value represents a variable’s contribution (loading) to a given PCA dimension. Values range from −1 to +1; larger absolute values indicate stronger contribution. Notable loadings per component:

Component	Dominant Variables	Interpretation
PC1	`pl_orbsmax` (0.810), `pl_orbper` (0.386)	Orbital size and period
PC2	`st_logg` (0.442), `st_rad` (−0.405), `pl_eqt` (−0.369)	Stellar–thermal gradient
PC3	`pl_orbeccen` (0.941)	Orbital eccentricity
PC4	`st_met` (0.830), `pl_bmasse` (0.292)	Metallicity and planetary mass

Model Artifacts

preprocessing_pipeline.joblib

Fitted sklearn pipeline containing three sequential steps applied in order:

Log1p Column Transformer

Applies log1p(x) = log(1 + x) to the six positively skewed variables: pl_orbper, pl_orbsmax, pl_rade, pl_bmasse, pl_insol, st_rad.

RobustScaler

Centers and scales all 12 features using median and interquartile range. Applied before imputation so that KNN distance calculations are not dominated by variables with large absolute ranges.

KNNImputer

Fills remaining missing values using the 5 nearest neighbors weighted by distance (n_neighbors=5, weights='distance').

import joblib

pipeline = joblib.load('data/processed/preprocessing_pipeline.joblib')

# Apply to new data (must have all 12 FEATURE_COLUMNS in the correct order)
X_transformed = pipeline.transform(new_df[FEATURE_COLUMNS])

pca/pca_model.joblib

Fitted sklearn PCA object configured for 4 components with random_state=42. The model was fitted on data that was already scaled by the preprocessing pipeline (DATA_ALREADY_SCALED=True), so no additional scaling should be applied before calling transform.

import joblib

pca = joblib.load('data/processed/pca/pca_model.joblib')

# X_transformed: output of preprocessing_pipeline.transform()
X_pca = pca.transform(X_transformed)  # shape: (n_planets, 4)

The four output columns correspond to PC1, PC2, PC3, and PC4 in that order. The mean reconstruction error for the 731-planet training set is 0.066834.

Metadata Contracts

preprocessing_metadata.json

Documents the full pipeline configuration and row counts in a machine-readable contract. Downstream stages should read this file to validate that their input matches expectations.

{
  "source": "NASA Exoplanet Archive - Planetary Systems (PS)",
  "id_column": "pl_name",
  "feature_columns": ["pl_orbper", "pl_orbsmax", "pl_rade", "pl_bmasse",
                       "pl_orbeccen", "pl_insol", "pl_eqt",
                       "st_teff", "st_rad", "st_mass", "st_met", "st_logg"],
  "log1p_features": ["pl_orbper", "pl_orbsmax", "pl_rade", "pl_bmasse", "pl_insol", "st_rad"],
  "minimum_present_features": 8,
  "scaler": "robust",
  "imputer": {"name": "KNNImputer", "n_neighbors": 5, "weights": "distance"},
  "rows_raw": 910,
  "rows_retained": 731,
  "rows_excluded": 179
}

pca/pca_metadata.json

Documents the PCA configuration, the number of components selected, and the variance retained. Use this contract to verify that the PCA solution is consistent with any downstream pipeline that loads pca_model.joblib.

{
  "n_components": 4,
  "retained_variance": 0.90391113362679,
  "variance_threshold": 0.85,
  "random_state": 42,
  "data_already_scaled": true,
  "feature_columns": ["pl_orbper", "pl_orbsmax", "pl_rade", "pl_bmasse",
                       "pl_orbeccen", "pl_insol", "pl_eqt",
                       "st_teff", "st_rad", "st_mass", "st_met", "st_logg"]
}

When applying saved artifacts to new data, always use pipeline.transform() and pca.transform() — never re-fit. Re-fitting would recalculate scaler parameters and PCA loadings from the new data, making results incomparable to the original 731-planet clustering solution.

Overview

Concepts

Pipeline Notebooks

Results & Interpretation

Reference

Pipeline Artifacts: All Files Produced by ExoProfiler

Directory Layout

Core Data Artifacts

exoplanets_preprocessed.csv

exoplanets_selected_raw.csv

clustered_exoplanets.csv

pca/pca_scores.csv

Audit Artifacts

preprocessing_row_audit.csv

preprocessing_quality_summary.csv

preprocessing_scaler_comparison.csv

preprocessing_imputer_comparison.csv

pca/pca_explained_variance.csv

pca/pca_loadings.csv

Model Artifacts

preprocessing_pipeline.joblib

pca/pca_model.joblib

Metadata Contracts

preprocessing_metadata.json

pca/pca_metadata.json

Build docs developers (and LLMs) love

Overview

Concepts

Pipeline Notebooks

Results & Interpretation

Reference

Documentation Index

​Directory Layout

​Core Data Artifacts

​exoplanets_preprocessed.csv

​exoplanets_selected_raw.csv

​clustered_exoplanets.csv

​pca/pca_scores.csv

​Audit Artifacts

preprocessing_row_audit.csv

preprocessing_quality_summary.csv

preprocessing_scaler_comparison.csv

preprocessing_imputer_comparison.csv

​pca/pca_explained_variance.csv

​pca/pca_loadings.csv

​Model Artifacts

​preprocessing_pipeline.joblib

​pca/pca_model.joblib

​Metadata Contracts

​preprocessing_metadata.json

​pca/pca_metadata.json

Build docs developers (and LLMs) love

Directory Layout

Core Data Artifacts

exoplanets_preprocessed.csv

exoplanets_selected_raw.csv

clustered_exoplanets.csv

pca/pca_scores.csv

Audit Artifacts

pca/pca_explained_variance.csv

pca/pca_loadings.csv

Model Artifacts

preprocessing_pipeline.joblib

pca/pca_model.joblib

Metadata Contracts

preprocessing_metadata.json

pca/pca_metadata.json