Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/LauraSilRu/exoplanet-profiler/llms.txt

Use this file to discover all available pages before exploring further.

ExoProfiler is an unsupervised machine learning pipeline that analyzes physical, orbital, and stellar characteristics of exoplanets discovered by NASA’s TESS mission to reveal hidden structure in astronomical data. Rather than relying on manually defined categories, it applies dimensionality reduction and clustering to discover natural groupings — called planetary families — purely from the numbers. It is designed for data scientists, astronomers, and researchers who want a reproducible, auditable approach to large-scale exoplanet classification.

​
Motivation

The pace of space exploration means that catalogs of known exoplanets now contain hundreds to thousands of objects, each described by dozens of physical measurements. Manually reviewing these datasets to find meaningful patterns is impractical, and the diversity of planet types — from rocky super-Earths to hot gas giants — makes rule-based classification brittle. Unsupervised learning offers a complementary path: instead of imposing categories top-down, algorithms find the groupings that actually exist in the data. This is particularly valuable for exoplanet science, where detection biases are significant. Transit photometry (the method used by TESS) preferentially finds large planets on short orbits, meaning any human-drawn classification scheme may unconsciously reflect instrument sensitivity rather than genuine planetary diversity. By letting the data drive the segmentation, ExoProfiler surfaces structure that is grounded in the measurements themselves. A further advantage is interpretability: once clusters are formed, profiling their feature distributions produces descriptive “families” that connect mathematical groupings back to astrophysical meaning — making the results useful both for machine learning practitioners and for domain experts.

​
What ExoProfiler Does

The pipeline is organized into six sequential stages across five Jupyter notebooks, each producing versioned artifacts consumed by the next stage.
1

Data Understanding

Load the raw NASA Exoplanet Archive TESS extract (910 rows, 355 columns) and audit variable types, null rates, and domain validity. Identify pl_name as the unique identifier and agree on 12 candidate feature columns for subsequent stages.
2

Exploratory Data Analysis (EDA)

Examine distributions, correlations, and outlier patterns across the 12 candidate variables. Confirm which features are highly skewed, characterize missing-data structure, and validate the physical plausibility of extreme values. These findings directly inform preprocessing decisions.
3

Preprocessing

Apply log1p transformations to six positively skewed variables, scale all 12 with RobustScaler, and impute missing values with a KNNImputer (5 neighbors, distance weights). A row-level completeness filter requiring at least 8 of 12 observed features retains 731 exoplanets. The fitted pipeline is serialized to preprocessing_pipeline.joblib for reuse on new data.
4

PCA

Reduce the preprocessed 12-variable matrix to 4 principal components, which together capture 90.35% of total variance — exceeding the 85% retention threshold. PC1 summarizes orbital scale, PC2 captures a stellar–thermal gradient, PC3 isolates orbital eccentricity, and PC4 preserves metallicity signal. The fitted model and component scores are exported to data/processed/pca/.
5

Clustering & Evaluation

Run K-Means on the 4 PCA components for K = 2 through 10, selecting K = 2 based on a clear Silhouette peak (0.4950). Stability is verified by re-running with four different random seeds, yielding 99.04%–100.00% label agreement. Hierarchical clustering independently achieves 97.13% agreement, and DBSCAN corroborates the same minority outlier population.
6

Cluster Profiling

Assign each of the 731 planets a numeric label (Cluster_K2) and a human-readable family name (Familia_Planeta). Profile each cluster by computing feature-level statistics in the original unscaled space and link the mathematical groupings back to astrophysical interpretation. Results are exported to clustered_exoplanets.csv.

​
Key Outcomes

731 Exoplanets Analyzed

Starting from 910 raw TESS records, 731 exoplanets (80.33%) passed the completeness filter requiring at least 8 of 12 features observed. Each planet is described by 12 physical, orbital, and stellar variables.

4 PCA Components — 90.35% Variance

Principal Component Analysis compressed the 12-variable space into 4 orthogonal components while retaining 90.35% of total variance, well above the 85% threshold. The reconstruction error across all 731 planets is 0.066834.

K=2 Clusters — Silhouette 0.4950, 99%+ Stability

K-Means identifies two clusters as the primary structure. The Silhouette coefficient of 0.4950 represents the clear maximum across K = 2–10. Multi-seed stability tests confirm 99.04%–100.00% label reproducibility.

Two Planetary Families

Standard Population (Cluster 0): 667 planets — 91.24% of the sample. Captures the continuum of rocky planets, super-Earths, and sub-Neptunes. Exotic Objects (Cluster 1): 64 planets — 8.76%. Isolates massive gas giants and hot Jupiters with extreme orbital parameters.

​
Technologies

ExoProfiler is built entirely in Python and relies on the following open-source libraries:
  • Python — core language for all notebooks, source modules, and the interactive app
  • Pandas — tabular data loading, filtering, and feature engineering
  • NumPy — numerical array operations and transformation utilities
  • Scikit-learn — preprocessing pipeline (RobustScaler, KNNImputer), PCA, K-Means, Silhouette scoring, and DBSCAN
  • Matplotlib — static plots including scree plots, loading heatmaps, and cluster distribution charts
  • Seaborn — statistical visualization of feature distributions and correlation matrices
  • Plotly — interactive 2D and 3D scatter plots of PCA projections colored by cluster
  • Streamlit — interactive web application for exploring planetary families
  • Jupyter Notebook / JupyterLab — execution environment for the five-stage pipeline notebooks

​
Project Structure

exoplanet-profiler/
├── data/
│   ├── raw/                    # Raw input data
│   └── processed/              # Preprocessed data and artifacts
│       └── pca/                # PCA model and scores
├── notebooks/
│   ├── 01_data_understanding.ipynb
│   ├── 02_eda.ipynb
│   ├── 03_preprocessing.ipynb
│   ├── 04_pca_and_cluster_profiling.ipynb
│   └── 05_clustering_and_evaluation.ipynb
├── src/
│   ├── preprocessing.py
│   ├── clustering.py
│   └── visualization.py
├── app/
│   └── app.py
├── docs/
├── requirements.txt
└── README.md
The cluster labels produced by ExoProfiler — “Standard Population” and “Exotic Objects” — represent mathematical structures discovered in the data and are not official astronomical categories. They reflect patterns in the TESS subset at the time of analysis and should be interpreted in light of known detection biases, the high imputation rate for certain variables (notably pl_insol at ~52%), and the completeness filter that under-represents recently discovered planets.

Build docs developers (and LLMs) love

Get started for freeTalk to us