Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/LauraSilRu/exoplanet-profiler/llms.txt

Use this file to discover all available pages before exploring further.

ExoProfiler transforms raw NASA TESS measurements into interpretable planetary families through a six-stage reproducible pipeline. Each stage produces validated artifacts that feed the next, ensuring the workflow is fully auditable.

​
Pipeline Stages

1

Stage 1 — Data Understanding (notebook 01)

Load the raw CSV and inspect its full 355-column structure. Identify the 12 candidate features that carry scientifically meaningful orbital, size, thermal, and stellar signals. Check data types, null rates, and verify that pl_name is unique and complete for use as a row identifier.
2

Stage 2 — Exploratory Data Analysis (notebook 02)

Examine distributions and inter-variable correlations across the candidate features. Perform an outlier audit using the interquartile range (IQR) as a descriptive tool. Document the scientific rationale for keeping all 12 features, noting that strong correlations are not a reason for exclusion when PCA follows downstream.
3

Stage 3 — Preprocessing (notebook 03)

Apply a completeness filter requiring each planet to have at least 8 of 12 features observed. Transform positively-skewed variables with log1p, scale all 12 variables with RobustScaler, and fill remaining gaps with KNNImputer(n_neighbors=5, weights="distance"). Output: a clean 731 × 12 numeric matrix with zero nulls and zero infinities.
4

Stage 4 — PCA (notebook 04)

Fit a full 12-component PCA on the preprocessed matrix. Select the fewest components whose cumulative explained variance meets the 85% threshold. Four components are required, retaining 90.35% of total variance. Output: pca_scores.csv with 731 rows and 4 principal component columns.
5

Stage 5 — Clustering (notebook 05)

Run a K-Means sweep over K = 2–10. Select K = 2 based on the Silhouette peak of 0.4950. Validate stability by varying random seeds (99.04%–100.00% agreement) and cross-check with DBSCAN and hierarchical clustering (97.13% agreement). Output: clustered_exoplanets.csv with cluster labels for all 731 planets.
6

Stage 6 — Cluster Profiling

Merge the two cluster labels back with the original (unscaled) feature values. Characterize each planetary family’s physical properties — orbital architecture, size distribution, thermal regime, and host-star characteristics — to produce interpretable “World Families” profiles.

​
Data Flow

The following diagram traces how files move between stages and where each artifact is written to disk. 
data/raw/exoplanets.csv
        │
        ▼
01_data_understanding.ipynb  ──►  candidate feature list
        │
        ▼
02_eda.ipynb  ──►  feature selection rationale
        │
        ▼
03_preprocessing.ipynb
        │
        ├──► data/processed/exoplanets_preprocessed.csv  (731 × 12)
        ├──► data/processed/preprocessing_pipeline.joblib
        └──► data/processed/preprocessing_metadata.json
                │
                ▼
04_pca_and_cluster_profiling.ipynb
        │
        ├──► data/processed/pca/pca_scores.csv  (731 × 4)
        ├──► data/processed/pca/pca_model.joblib
        └──► data/processed/pca/pca_metadata.json
                │
                ▼
05_clustering_and_evaluation.ipynb
        │
        └──► data/processed/clustered_exoplanets.csv  (731 × 7)

​
Design Principles

Unsupervised by design. No labels, categories, or predefined planet types are used at any stage. Every grouping emerges entirely from the data’s own structure.
Full reproducibility. All fitted transformers — the preprocessing pipeline, PCA model, and cluster labels — are serialized as .joblib artifacts. To score a new planet, load the saved pipeline and PCA model; do not refit on a single observation.
Outlier preservation. Physical outliers such as extreme hot Jupiters are real objects, not noise. Rather than deleting or winsorizing them, ExoProfiler applies log1p transformations and RobustScaler (median + IQR) to limit their leverage on the model without discarding the information they carry.
Imputation risk for pl_insol. Even after the completeness filter, insolation flux is missing in a large fraction of retained rows (51.71% imputed within the 731-planet sample). Cluster interpretations that emphasize insolation flux should be treated with additional caution.

​
Execution Order

Run the notebooks in the following order. Each notebook assumes all prior artifacts are present.
  1. 01_data_understanding.ipynb
  2. 02_eda.ipynb
  3. 03_preprocessing.ipynb
  4. 04_pca_and_cluster_profiling.ipynb
  5. 05_clustering_and_evaluation.ipynb

Build docs developers (and LLMs) love

Get started for freeTalk to us