Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/LauraSilRu/exoplanet-profiler/llms.txt

Use this file to discover all available pages before exploring further.

04_pca_and_cluster_profiling.ipynb reduces the 12-dimensional preprocessed feature space to 4 principal components that collectively retain 90.35% of total variance. These 4 components serve as the input matrix for K-Means clustering in notebook 05. The notebook does not fit or evaluate any clustering model — its sole responsibility is dimensionality reduction and the export of PCA artifacts.

​
What PCA Does

PCA (Principal Component Analysis) creates new variables — called principal components — that are orthogonal linear combinations of the original 12 features. PC1 is constructed to capture the maximum possible variance in the data. PC2 captures the maximum remaining variance while being perpendicular (uncorrelated) to PC1. Each subsequent component follows the same rule. Because the data is already scaled by RobustScaler in notebook 03, PCA is applied directly without any additional rescaling. Re-scaling would overwrite the carefully chosen robust statistics and break the pipeline contract established in notebook 03. PCA is an unsupervised technique — it has no knowledge of clusters or labels. It reduces dimensionality and removes correlated redundancy, but it does not determine how many groups exist in the data.

​
Configuration

USE_DEMO_DATA = False
PROCESSED_DATA_PATH = Path('data/processed/exoplanets_preprocessed.csv')
DATA_ALREADY_SCALED = True
VARIANCE_THRESHOLD = 0.85  # select fewest components reaching 85%
pl_name is used only as a row identifier and is excluded from the PCA matrix. The VARIANCE_THRESHOLD of 0.85 sets the minimum acceptable cumulative explained variance — the notebook selects the fewest components that meet or exceed this threshold.

​
Component Selection

PCA is first fitted with all 12 possible components to obtain the full variance profile. Then the minimum number of components whose cumulative variance reaches 85% is selected.
ComponentIndividual VarianceCumulative Variance
PC144.65%44.65%
PC229.56%74.21%
PC39.41%83.62%
PC46.74%90.35%
Three components reach only 83.62%, which falls below the 85% threshold. Four components are therefore selected, retaining 90.35% of the total variance. The variance profile is displayed as a scree plot whose purpose is solely to justify the component count — it is not used for clustering decisions.

​
Reconstruction Error

With 4 components applied to 731 observations across 12 features, the mean reconstruction error is 0.066834 (measured in the scaled space). This confirms that the 4-component approximation faithfully reproduces the preprocesed feature matrix with minimal information loss.

​
Loading Interpretations

Loadings indicate how much each original variable contributes to each principal component. Magnitude determines importance; relative signs within a component indicate whether variables move together or in opposite directions. PC1 — Orbital Dimension The first component is dominated by orbital geometry. The leading contributors are pl_orbsmax (0.816), pl_orbper (0.386), and pl_orbeccen (0.219). PC1 primarily summarizes orbital size and period — planets scoring high on PC1 tend to have large, long-period orbits. PC2 — Stellar-Thermal Gradient The second component contrasts stellar surface gravity against stellar size and the thermal environment of the planet. Key contributors: st_logg (0.442), st_rad (−0.405), pl_eqt (−0.369), and st_mass (−0.362). High PC2 scores indicate compact, high-gravity host stars; low PC2 scores indicate large, cool host stars with cooler planets. PC3 — Eccentricity The third component is almost entirely determined by orbital eccentricity: pl_orbeccen loads at 0.941, with minor contributions from pl_orbsmax (−0.240) and pl_bmasse (0.152). This component isolates the subset of planets on notably non-circular orbits. PC4 — Metallicity and Planetary Size The fourth component preserves stellar metallicity information that was not captured by the first three components. Contributors: st_met (0.830), pl_bmasse (0.292), and pl_rade (0.265). Without this component, metallicity — a key stellar property — would be largely absent from the clustering input.

​
Sign Convention

The global sign of any PCA component is mathematically arbitrary. Multiplying all loadings and all scores of a component by −1 produces an equally valid solution. Therefore, do not interpret a positive or negative sign in isolation. Interpret only the relative signs and magnitudes within a single component to understand which variables move together and which oppose each other.

​
Visualizations

The notebook produces two spatial projections for visual inspection:
  • 2D projection (PC1–PC2): Accounts for 74.21% of variance. Useful for identifying the primary structure separating orbital families from thermally distinct populations.
  • 3D projection (PC1–PC2–PC3): Accounts for 83.62% of variance. Adds the eccentricity dimension, revealing a small subset of high-eccentricity outliers.
Both projections are for exploratory visualization only. K-Means in notebook 05 operates on all 4 components, not just the 2 or 3 that can be plotted. When cluster labels from notebook 05 are available in data/processed/cluster_labels.csv, the notebook can join them by pl_name and color-code the projections for interpretation.

​
Output Artifacts

All files are written to data/processed/pca/.
FileDescription
pca/pca_scores.csv731 × 4 component scores: pl_name, PC1, PC2, PC3, PC4
pca/pca_explained_variance.csvPer-component individual and cumulative variance breakdown
pca/pca_loadings.csvVariable contribution (loading) per component
pca/pca_model.joblibFitted PCA model — apply to new data after the preprocessing pipeline
pca/pca_metadata.jsonContract: n_components, retained_variance, feature_columns, random_state
To transform new planets, apply preprocessing_pipeline.joblib first, then pca_model.joblib, always preserving the 12-column order specified in FEATURE_COLUMNS. 
import joblib
import pandas as pd

pipeline = joblib.load('data/processed/preprocessing_pipeline.joblib')
pca = joblib.load('data/processed/pca/pca_model.joblib')

X_scaled = pipeline.transform(new_df[FEATURE_COLUMNS])
scores = pca.transform(X_scaled)  # never call pca.fit_transform() on new data
Always call pca.transform() on new data — never pca.fit_transform(). Re-fitting would recompute the principal axes from the new batch, producing component scores that are incompatible with the K-Means centroids trained in notebook 05.

​
Running the Notebook

jupyter notebook notebooks/04_pca_and_cluster_profiling.ipynb
Prerequisite: data/processed/exoplanets_preprocessed.csv must exist and contain 731 rows with zero nulls and zero infinities. Run notebook 03 first if this file is missing or outdated.

Build docs developers (and LLMs) love

Get started for freeTalk to us