Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/NASAARSET/PM2.5_AQ_Online_2026/llms.txt

Use this file to discover all available pages before exploring further.

This homework notebook adapts the full PM₂.₅ satellite analysis workflow — introduced in the Case Study — to a new region and time period that you configure yourself. You will study New Delhi, India during the pre-monsoon and early monsoon period of June through August 2019, comparing SatPM V6.GL.03 monthly estimates and MERRA-2 CNN hourly estimates against reference monitors retrieved from OpenAQ. The structure mirrors the case study exactly, but key domain parameters are replaced by ???? placeholders that you fill in as part of the assignment.
This notebook requires the same credentials as the case study: a NASA Earthdata account and an OpenAQ API key. Unlike the case study, no SINCA scraper is needed — OpenAQ has good coverage for Indian reference monitors.

​
Open in Google Colab

🇺🇸 Exercise (EN)

English homework notebook — fill in the ???? placeholders

🇪🇸 Ejercicio (ES)

Cuaderno de tarea en español
Complete (answer-key) versions of the homework notebook are not publicly distributed. Check with your course instructor for access.

​
What the ???? placeholders mean

The exercise notebook ships with several cells containing literal ???? tokens. These are intentional gaps that you must replace with correct Python values before the cell will execute successfully. The table below lists every placeholder and the expected answer.
Placeholder locationVariableValue to fill in
Domain definitionn_lat_min27.5
Domain definitionn_lat_max30
Domain definitionn_lon_min76
Domain definitionn_lon_max78.5
Domain definitiont_startnp.datetime64('2019-06-01 00:00:00')
Domain definitiont_endnp.datetime64('2019-08-31 23:59:59')
Domain definitions_timezome'Asia/Kolkata'
Threshold analysisn_threshold50 (µg/m³)
Python will raise a SyntaxError or NameError if ???? is left in any executed cell. Make sure to replace all occurrences before running the notebook end-to-end.

​
Notebook walkthrough

1
Background
2
The notebook opens with the same framing as the case study: this is part of the NASA ARSET training on estimating surface PM₂.₅ from satellite data. The background cell notes that this version is intended for the Homework Assignment, signalling that certain cells are intentionally incomplete.
3
New Delhi is one of the world’s most PM₂.₅-polluted mega-cities. The June–August 2019 window captures the transition from the hot, dusty pre-monsoon season into early monsoon, providing a range of concentration regimes for students to explore.
4
Setup: install packages, mount Drive, and log in
5
Identical to the case study setup cell. All the same libraries are installed and imported, Google Drive is mounted, and Earthdata / OpenAQ credentials are collected.
6
# Install & Import the required packages:
!pip install --quiet cartopy earthaccess

import os, getpass
from google.colab import drive
import warnings; warnings.filterwarnings('ignore')

import numpy as np
import pandas as pd
import xarray as xr

import earthaccess, requests, urllib, re, time
from urllib.parse import urljoin
from bs4 import BeautifulSoup

import matplotlib.pyplot as plt
import cartopy
import seaborn as sns

drive.mount('/content/drive')
earthaccess.login()
s_key_openAQ = getpass.getpass('Enter OpenAQ API key:')
7
Define useful functions
8
The same four helper functions from the case study are defined verbatim:
9
  • F_subset_and_combine — opens, clips, and concatenates NetCDF files
  • F_get_OpenAQ_from_API — queries OpenAQ v3 for reference monitors
  • F_plot_map — renders PM₂.₅ data on a Cartopy map
  • F_compute_metrics — computes Bias, RMSE, MNB, MNRMSE, R², spatial and temporal correlations (see /api/f-compute-metrics)
    • 10
    No SINCA functions are defined — India does not have an equivalent publicly scrapeable national network, and OpenAQ provides adequate reference-monitor coverage for New Delhi.
    11
    Define the domain of interest
    12
    This is the first cell containing ???? placeholders. You must supply the bounding box coordinates, time range, and timezone for New Delhi.
    13
    # Define the bounding box:
n_lat_min = ????   # → 27.5
n_lat_max = ????   # → 30
n_lon_min = ????   # → 76
n_lon_max = ????   # → 78.5

# Define the start and end time:
t_start = np.datetime64('YYYY-MM-DD hh:mm:ss')  # → '2019-06-01 00:00:00'
t_end   = np.datetime64('YYYY-MM-DD hh:mm:ss')  # → '2019-08-31 23:59:59'

# Specify the local time zone:
s_timezome = '????'  # → 'Asia/Kolkata'

# Preview the selected region:
F_plot_map(n_lat_min=n_lat_min, n_lat_max=n_lat_max,
           n_lon_min=n_lon_min, n_lon_max=n_lon_max)
    14
    Once filled in correctly, F_plot_map will render a blank map of the New Delhi region so you can visually confirm the bounding box is correct before downloading any data.
    15
    India Standard Time (IST) is UTC+5:30. The TZ database identifier is 'Asia/Kolkata'. You can verify this with Python: import pytz; pytz.timezone('Asia/Kolkata').
    16
    Download SatPM data
    17
    Identical logic to the case study: the notebook loops over months in the analysis window, constructs the SatPM S3 URL for each month, downloads the file if not already present, and calls F_subset_and_combine to produce a cropped xarray dataset.
    18
    s_folder_SatPM = '/content/drive/MyDrive/Colab Notebooks/ARSET/PM25_Training_2026/SatPM_Data'

l_files_SatPM = []
for t_month in np.arange(t_start.astype('datetime64[M]'),
                          t_end.astype('datetime64[M]') + np.timedelta64(1,'M'),
                          np.timedelta64(1,'M')):
    s_year  = str(t_month)[0:4]
    s_month = str(t_month)[5:7]
    s_url = (f'https://s3.amazonaws.com/satpmdata/V6GL03/FineResolution/GL/'
             f'Monthly/{s_year}/V6GL03.CNNPM25.GL.{s_year}{s_month}-{s_year}{s_month}.nc')
    s_download_path = os.path.join(s_folder_SatPM, s_url.split('/')[-1])
    if not os.path.exists(s_download_path):
        urllib.request.urlretrieve(s_url, s_download_path)
    l_files_SatPM.append(s_download_path)

def F_time_from_SatPM_url(s_url):
    s_YYYYMM = s_url.split('.')[-2].split('-')[0]
    return np.datetime64(f'{s_YYYYMM[0:4]}-{s_YYYYMM[4:6]}-01 00:00:00')

a_data_SatPM = F_subset_and_combine(
    l_files_SatPM, n_lon_min, n_lat_min, n_lon_max, n_lat_max,
    t_start, t_end, F_time_from_file=F_time_from_SatPM_url)

a_data_SatPM['PM25_filtered'] = a_data_SatPM['PM25'].where(a_data_SatPM['PM25'] > 0)
a_data_SatPM.to_netcdf(os.path.join(s_folder_SatPM, 'SatPM_data_subset_homework.nc'))
    19
    The subsequent plot cell uses Colorbar_Limits=[0, 60] — note this is higher than the case study default of [0, 40], reflecting the higher baseline PM₂.₅ concentrations expected in the New Delhi region.
    20
    Download MERRA-2 CNN data
    21
    The earthaccess search-and-download pattern is identical to the case study. The QFLAG >= 3 filter is applied to retain only reliable hourly retrievals, and results are saved to MERRA2_CNN_data_subset_homework.nc.
    22
    s_folder_MERRA2_CNN = '/content/drive/MyDrive/Colab Notebooks/ARSET/PM25_Training_2026/MERRA2_CNN_Data'

l_results = earthaccess.search_data(
    short_name='MERRA2_CNN_HAQAST_PM25',
    bounding_box=(n_lon_min, n_lat_min, n_lon_max, n_lat_max),
    temporal=(str(t_start), str(t_end))
)
l_files_MERRA2_CNN = earthaccess.download(l_results, s_folder_MERRA2_CNN)

a_data_MERRA2_CNN = F_subset_and_combine(
    l_files_MERRA2_CNN, n_lon_min, n_lat_min, n_lon_max, n_lat_max, t_start, t_end)

a_data_MERRA2_CNN['PM25_filtered'] = (
    a_data_MERRA2_CNN['MERRA2_CNN_Surface_PM25']
    .where(a_data_MERRA2_CNN['QFLAG'] >= 3))

a_data_MERRA2_CNN.to_netcdf(os.path.join(s_folder_MERRA2_CNN,
                                          'MERRA2_CNN_data_subset_homework.nc'))
    23
    Download monitor data from OpenAQ
    24
    For India, OpenAQ is the primary and only ground-truth source in this notebook. Reference monitors (i.e., b_reference=True, b_lowcost=False) are queried for the bounding box and time window. There is no SINCA-vs-OpenAQ comparison step — the notebook proceeds directly with OpenAQ data.
    25
    s_folder_Main = '/content/drive/MyDrive/Colab Notebooks/ARSET/PM25_Training_2026'

f_data_monitors = F_get_OpenAQ_from_API(
    n_lon_min, n_lat_min, n_lon_max, n_lat_max, t_start, t_end)

# Filter out physically implausible values:
f_data_monitors['PM25_filtered'] = f_data_monitors['PM2.5'].where(
    (f_data_monitors['PM2.5'] > 0) & (f_data_monitors['PM2.5'] < 1000))

f_data_monitors.to_csv(os.path.join(s_folder_Main, 'Monitor_data_subset_homework.csv'))
    26
    The upper filter is 1000 µg/m³ here (vs. 2000 µg/m³ in the Chile case study), reflecting the lower peak values expected in the New Delhi summer compared to a wildfire episode.
    27
    Analysis: maps
    28
    Two overlay maps are produced — SatPM and MERRA-2 CNN with OpenAQ monitor scatter points superimposed — using a colour scale of [0, 60] µg/m³ appropriate for South Asian background PM₂.₅ levels.
    29
    # Overlay monitors on the SatPM map:
F_plot_map(a_data_plot       = a_data_SatPM['PM25_filtered'],
           a_data_plot_point = f_data_monitors['PM25_filtered'],
           n_lat_min=n_lat_min, n_lat_max=n_lat_max,
           n_lon_min=n_lon_min, n_lon_max=n_lon_max,
           Colorbar_Label  = 'PM$_{2.5}$ [µg/m$^{3}$]',
           Plot_Title      = f'Comparing SatPM to Monitors\n(Average {t_start} to {t_end})',
           Colorbar_Limits = [0, 60])

# Overlay monitors on the MERRA-2 CNN map:
F_plot_map(a_data_plot       = a_data_MERRA2_CNN['PM25_filtered'],
           a_data_plot_point = f_data_monitors['PM25_filtered'],
           n_lat_min=n_lat_min, n_lat_max=n_lat_max,
           n_lon_min=n_lon_min, n_lon_max=n_lon_max,
           Colorbar_Label  = 'PM$_{2.5}$ [µg/m$^{3}$]',
           Plot_Title      = f'Comparing MERRA-2 CNN to Monitors\n(Average {t_start} to {t_end})',
           Colorbar_Limits = [0, 60])
    30
    Analysis: monthly averages vs. SatPM
    31
    Monitor data are aggregated to monthly averages (minimum 480 valid hourly samples per station-month) and the nearest SatPM and MERRA-2 CNN grid values are joined in. Scatter plots colour-coded by month and F_compute_metrics output provide the quantitative comparison.
    32
    Analysis: hourly averages vs. MERRA-2 CNN (with MERRA-2 gridding)
    33
    Monitors are snapped to the nearest MERRA-2 grid cell (0.5° lat × 0.625° lon) and averaged per grid cell per hour, with a minimum of 3 valid samples required. The gridded dataset is augmented with local_day and local_hour_of_day columns (using Asia/Kolkata timezone) for downstream analyses.
    34
    n_minimum_samples_per_grid = 3

f_data_monitors_gridded = f_data_monitors.copy().reset_index()
f_data_monitors_gridded['lat_grid'] = round(f_data_monitors_gridded['lat'] / 0.5)  * 0.5
f_data_monitors_gridded['lon_grid'] = round(f_data_monitors_gridded['lon'] / 0.625) * 0.625

f_data_combined_gridded = (
    f_data_monitors_gridded
    .groupby(['lat_grid', 'lon_grid', 'time'])[['PM25_filtered']]
    .count()
    .rename(columns={'PM25_filtered': 'Count'})
)
f_data_combined_gridded['Average'] = (
    f_data_monitors_gridded
    .groupby(['lat_grid', 'lon_grid', 'time'])[['PM25_filtered']].mean()
)
f_data_combined_gridded = (
    f_data_combined_gridded
    .where(f_data_combined_gridded['Count'] >= n_minimum_samples_per_grid)
    .dropna()
)
    35
    Analysis: diel cycle comparison
    36
    Median PM₂.₅ and the interquartile range (25th–75th percentile band) are plotted against local hour of day for both monitors (black) and MERRA-2 CNN (red). In New Delhi, the diel pattern is strongly influenced by traffic and boundary-layer dynamics, providing a meaningful test of MERRA-2 CNN’s diurnal accuracy.
    37
    Analysis: daily averages vs. MERRA-2 CNN
    38
    Gridded hourly values are further aggregated to local calendar days. A scatter plot coloured by MERRA-2 latitude grid value and F_compute_metrics output quantify agreement at the daily scale.
    39
    Analysis: threshold analysis
    40
    This cell contains the second ???? placeholder. You must set n_threshold to the value specified in the homework instructions.
    41
    # Set the desired threshold for analysis:
n_threshold = ????   # → 50  (µg/m³, as instructed for the homework)

f_data_combined_gridded['Daily_Average_Above_Threshold'] = False
for i_row in range(len(f_data_combined_gridded)):
    n_lat  = f_data_combined_gridded.index[i_row][0]
    n_lon  = f_data_combined_gridded.index[i_row][1]
    t_time = f_data_combined_gridded.index[i_row][2]
    t_day  = f_data_combined_gridded.loc[(n_lat, n_lon, t_time), 'local_day']
    if f_data_combined_gridded_daily.loc[(n_lat, n_lon, t_day), 'Average'] > n_threshold:
        f_data_combined_gridded.loc[(n_lat, n_lon, t_time),
                                    'Daily_Average_Above_Threshold'] = True

# Performance metrics for days below the threshold:
print('Performance Metrics for MERRA-2 CNN (values below threshold):')
F_compute_metrics(
    f_data_combined_gridded[
        f_data_combined_gridded['Daily_Average_Above_Threshold'] == False],
    s_time='time', s_value='Average', s_estimate='MERRA2_CNN')
    42
    The threshold of 50 µg/m³ corresponds to the Indian National Ambient Air Quality Standard (NAAQS) 24-hour average limit for PM₂.₅. Days above this threshold are classified as “polluted”; the metrics below threshold show how well the models capture relatively cleaner conditions.
    43
    Analysis: time series for key locations
    44
    Three New Delhi grid cells are pre-populated in the homework notebook. The initial active location is set to 'New Delhi (East)', but you can change s_location to explore any of the three.
    45
    # Pre-populated MERRA-2 grid locations for New Delhi:
d_location = {
    'New Delhi (East)':  (28.5, 77.5),
    'New Delhi (West)':  (28.5, 76.875),
    'New Delhi (North)': (29.0, 76.875),
}
s_location = 'New Delhi (East)'

f_ts = f_data_combined_gridded_daily.loc[d_location[s_location]]

o_fig = plt.figure(figsize=(15, 5))
o_ax  = o_fig.add_subplot()
o_ax.plot(f_ts.index, f_ts['Average'],    color='black', label='Monitor Average')
o_ax.plot(f_ts.index, f_ts['MERRA2_CNN'], color='red',   label='MERRA-2 CNN Average')
o_ax.legend()
o_ax.set_ylabel('Daily Average PM$_{2.5}$ [µg/m$^{3}$]')
o_ax.set_title(f'Location: {s_location}')

    ​
    Summary of ???? placeholders

    CellVariableCorrect value
    Domain definitionn_lat_min27.5
    Domain definitionn_lat_max30
    Domain definitionn_lon_min76
    Domain definitionn_lon_max78.5
    Domain definitiont_startnp.datetime64('2019-06-01 00:00:00')
    Domain definitiont_endnp.datetime64('2019-08-31 23:59:59')
    Domain definitions_timezome'Asia/Kolkata'
    Threshold analysisn_threshold50

    ​
    Key concepts and cross-references

    ConceptDetails
    Ground-truth networkOpenAQ — global aggregator of reference PM₂.₅ monitors; provides hourly validated records for India
    Monthly satellite productSatPM V6.GL.03 — 0.01° resolution, global monthly PM₂.₅
    Hourly satellite productMERRA-2 CNN (DOI: 10.5067/OCKK5HCFW5N3) — 0.5° × 0.625°, hourly
    Performance metricsF_compute_metrics — Bias, RMSE, MNB, MNRMSE, R², spatial and temporal correlations
    MERRA-2 grid snappingSnap to nearest 0.5° lat × 0.625° lon grid centre; minimum 3 valid samples per grid cell per hour
    TimezoneAsia/Kolkata (IST = UTC+5:30) for diel-cycle and daily-average calculations
    Compare your New Delhi results to the Chile case study. Are the metrics better or worse? What might explain differences — density of reference monitors, concentration range, seasonal vs. episodic pollution patterns?

    Build docs developers (and LLMs) love

    Get started for freeTalk to us