Use this file to discover all available pages before exploring further.
F_compute_metrics produces a standardized set of statistics for comparing a modeled or satellite-derived PM2.5 estimate against ground-level monitor observations. All metrics are computed over the paired (estimate, measured) record that results from merging gridded model output with grid-snapped monitor averages at matching time steps and locations.Call the function after constructing a combined DataFrame that contains both the monitor-derived Average column and the model column (e.g., MERRA2_CNN):
print('Performance Metrics for MERRA-2 CNN (daily)')F_compute_metrics(f_data_combined_gridded_daily, s_time='local_day', s_value='Average', s_estimate='MERRA2_CNN')
Parameter
Meaning
s_time
Column name used to identify the time dimension (e.g., 'local_day', 'time')
The arithmetic mean of all estimate values and all measured values in the paired record, respectively.
Estimated Mean = mean(estimate)Measured Mean = mean(measured)
Interpretation: Comparing these two scalars gives the first-order sense of whether the model runs high or low overall. A large gap without normalizing indicates systematic over- or under-prediction.
Units: Same as PM2.5 (µg/m³).Interpretation: A positive bias means the model over-predicts on average; a negative bias means it under-predicts. Bias is sensitive to outliers in the same direction and does not capture random scatter.
Bias and Measured Mean together determine the sign and magnitude of Mean Normalized Bias. Always report Bias alongside its normalized counterpart so readers can judge whether a large MNB stems from a small measured mean or a genuinely large systematic error.
Units: Dimensionless (often expressed as a percentage by multiplying by 100).Interpretation: Normalizing by the measured mean makes MNB comparable across regions and seasons with different background PM2.5 levels. The US EPA performance goal for regional photochemical models is |MNB| ≤ 0.15 (±15 %).
Units: µg/m³.Interpretation: RMSE penalizes large individual errors more than mean absolute error does (because of the squaring). It captures both systematic bias and random scatter. Lower is better; values near zero indicate the model tracks individual observations closely.
Units: Dimensionless.Interpretation: Like MNB, normalizing RMSE by the measured mean allows comparison across domains. A MNRMSE of 0.5, for example, means the typical error magnitude is 50 % of the average observed concentration. The EPA performance goal is MNRMSE ≤ 0.35 (35 %).
Range: −∞ to 1. A perfect model yields R² = 1; a model no better than the mean of the observations yields R² = 0; a model worse than the mean yields R² < 0.Interpretation: R² describes the fraction of observed variance explained by the model. High R² with high Bias is common when a model captures temporal and spatial patterns but has a persistent offset — always examine R² and Bias together.
R² can be artificially inflated when the dataset spans a very wide dynamic range (e.g., combining a pristine rural region with a heavily polluted urban area). In that case, stratify the evaluation by region or season and interpret R² in the context of MNB and RMSE.
Spatial Correlation = median( r_spatial ) [min to max]
Interpretation: Spatial correlation tells you whether the model correctly reproduces the geographic pattern of PM2.5 at a given moment — high values where concentrations are high, low values where they are low. A high spatial correlation with low temporal correlation suggests the model captures mean spatial gradients but not day-to-day variability.
Temporal Correlation = median( r_temporal ) [min to max]
Interpretation: Temporal correlation tells you whether the model captures the time-varying behavior at a fixed site — pollution episodes, seasonal cycles, weekday/weekend patterns. Reporting the median plus min-to-max range across all locations reveals whether good performance at some sites masks poor performance elsewhere.
Running metrics on the above/below threshold subsets
# Days with daily average at or above thresholdf_above = f_data_combined_gridded_daily[ f_data_combined_gridded_daily['Average'] >= n_threshold]f_below = f_data_combined_gridded_daily[ f_data_combined_gridded_daily['Average'] < n_threshold]print(f'--- Days ≥ {n_threshold} µg/m³ ---')F_compute_metrics(f_above, s_time='local_day', s_value='Average', s_estimate='MERRA2_CNN')print(f'--- Days < {n_threshold} µg/m³ ---')F_compute_metrics(f_below, s_time='local_day', s_value='Average', s_estimate='MERRA2_CNN')
Comparing above/below results reveals whether the model has differential skill during pollution episodes — a critical consideration when the output will be used for regulatory or health-impact applications.
