Documentation Index
Fetch the complete documentation index at: https://mintlify.com/HelenDiMo/archaios-data-Intelligence/llms.txt
Use this file to discover all available pages before exploring further.
desembarco_normandia.ipynb is the analytical heart of the A.D.I. project. It documents every step of the data pipeline — from loading the five raw THOR CSV files through exploratory statistical analysis to producing the three ADI master datasets that power the Power BI dashboards. This page walks through each notebook section, explaining what the code does, what the outputs mean, and how each step feeds the archaeological prospection model for Normandy.
Running the Notebook
Install dependencies
Install the required Python libraries before opening the notebook:pip install jupyter pandas numpy matplotlib seaborn
Launch the notebook
From the repository root, launch Jupyter and open the analysis notebook:jupyter notebook desembarco_normandia.ipynb
Verify the data paths
The notebook expects the five THOR source CSV files to be present in the data/ subdirectory. The three master output CSVs (adi_dataset_master.csv, adi_master_1_5_junio.csv, and adi_master_1943_1944_CLEAN.csv) are written directly to the repository root once you run all cells in sequence.
The notebook is in Spanish. Variable names and markdown commentary are in Spanish, consistent with the project deliverable language. The underlying CSV column names remain in English (from the original THOR database).
Notebook Structure
The notebook is organised into four major sections that mirror the analytical workflow — from raw ingestion to final enrichment.
Section I: Data Exploration and Preparation
The opening section establishes the data inventory. All five THOR source files are loaded with pd.read_csv(..., low_memory=False) and each is profiled for dimensions, column data types, null counts, and a five-row preview. This provides the foundation for deciding which datasets are suitable for the Normandy-specific analysis.
| Dataset | File | Records | Columns | Role in A.D.I. |
|---|
| 1 | 1_cleanbombww2.csv | 1,109 | 21 | Scoped to MTO — excluded |
| 2 | 2_operations.csv | 178,281 | 46 | Primary analysis dataset |
| 3 | 3_thor_wwii_aircraft_gloss.csv | 52 | 6 | Aircraft code glossary |
| 4 | 4_thor_wwii_data_clean.csv | 178,281 | 62 | Full-theater cleaned THOR data |
| 5 | 5_thor_wwii_weapon_gloss.csv | 59 | 8 | Weapon and bomb type glossary |
1_cleanbombww2.csv (1,109 records, 21 columns): This file is a clean version of the MTO (Mediterranean Theater of Operations) dataset, primarily covering Italy and Sicily. When filtered to France in 1944, only one mission is returned, and its Target City coordinate resolves to [44°N 1°E] — central France, far from Normandy. The notebook concludes that this dataset carries an irrecoverable geographic bias for our purposes and excludes it from all further analysis.
Dataset 2 — 2_operations.csv (178,281 records, 46 columns): This is the primary dataset for the project. It contains digitised WWII mission reports from USAAF and RAF operations, covering every theater from 1939 to 1945. Each record includes mission date, theater, country, aircraft series, target location coordinates, bomb load by type, and total weight in tons. Filtered to France (1943–1944), it yields 17,795 missions — the backbone of the A.D.I. Normandy analysis.
Dataset 3 — 3_thor_wwii_aircraft_gloss.csv (52 records, 6 columns): A reference glossary with columns aircraft (short code), name, full_name, aircraft_type, and a hyperlink to a specification page. This table is joined to Dataset 2 to decode Aircraft Series short codes (e.g., B17, B26, A36) into human-readable model names (e.g., Boeing B-17 Flying Fortress) for the Power BI dashboard.
Dataset 4 — 4_thor_wwii_data_clean.csv (178,281 records, 62 columns): The expanded, fully cleaned THOR WWII dataset. It contains more columns than Dataset 2, including additional target classification fields (tgt_type, tgt_industry, tgt_priority) and sighting method codes. This dataset is used to audit target-type distributions across the full France/1944 scope.
Dataset 5 — 5_thor_wwii_weapon_gloss.csv (59 records, 8 columns): A weapon glossary covering all ordnance types appearing in the THOR records. Columns include weapon_name, weapon_type, weapon_class (HE, IC, FRAG), alt_weapon_name, and weapon_description. This glossary informs the weapon_type classification logic applied to the master dataset.
Section II: D-Day Statistical Analysis
Section II filters each dataset down to France-specific missions and runs descriptive statistical analysis. The code tests multiple date string formats to ensure complete capture of June 6, 1944 records (06/06/1944, 6/6/1944, 1944-06-06).
Dataset 1 findings (France, 1944): As expected from the Section I profile, the filter returns exactly 1 mission. Its Target City is 44 00 N 001 00 E — a raw coordinate string placed in the city field, mapping to central France. This confirms the dataset is unsuitable and is discarded.
Dataset 2 findings (France, June 6, 1944 — D-Day): 535 missions are identified. Top five target cities on D-Day by mission count:
| Rank | Target City | Missions |
|---|
| 1 | CAEN | 80 |
| 2 | UNKNOWN | 38 |
| 3 | VIRE | 28 |
| 4 | ARROMANCHES | 24 |
| 5 | ARGENTAN | 23 |
Total Weight (Tons) field for the 535 D-Day missions shows mean = 35.6t, median = 23t, max = 523t. The distribution is right-skewed, with high-tonnage strategic raids inflating the average.
Dataset 4 findings (France, June 6, 1944): The top target types for D-Day missions, drawn from the fuller tgt_type column available in Dataset 4:
| Rank | Target Type | Count |
|---|
| 1 | TACTICAL TARGETS | 111 |
| 2 | TACTICAL TARGET | 48 |
| 3 | MARSHALLING YARD | 25 |
| 4 | CITY AREA | 24 |
| 5 | BATTLE APEA | 21 |
Section III: Historical Context, France 1943–1944
Section III filters Dataset 2 to the full France scope (1943–1944), producing 17,795 missions — the backbone of the A.D.I. Normandy analysis. It generates the adi_master_1943_1944_CLEAN.csv master table and contextualises the statistical findings within the military history of the Allied air campaign.
Tonnage anomaly identified: The Total Weight (Tons) field shows a max of 999.0 tons — an outlier the notebook identifies as a legacy documentation placeholder (a sentinel value used when the exact figure was unknown at digitisation time). The full-period distribution is right-skewed: mean = 44.1t, median = 30t. The 999-ton value is flagged in the data governance documentation and excluded from tonnage-based heat maps.
Top five target cities (France, 1943–1944):
| Rank | Target City | Missions |
|---|
| 1 | BREST | 527 |
| 2 | ST LO | 404 |
| 3 | UNKNOWN | 322 |
| 4 | METZ | 279 |
| 5 | PAS DE CALAIS | 272 |
Aircraft Series ↔ aircraft, using a left join to preserve all missions even when the aircraft code has no glossary entry. A fallback column Avion_Nombre_Completo is created:
df_enriquecido['Avion_Nombre_Completo'] = (
df_enriquecido['full_name'].fillna(df_enriquecido['Aircraft Series'])
)
| Aircraft Code | Full Name | Missions |
|---|
| B17 | Boeing B-17 Flying Fortress | 5,278 |
| B26 | Martin B-26 Marauder | 3,991 |
| B24 | Consolidated B-24 Liberator | 3,625 |
| HVY | Heavy bomber (generic) | 1,161 |
| GB17 | British B-17 variant | 1,113 |
Section IV: June 1–5 Pre-D-Day Analysis
The final section isolates the five days immediately before the landings to document the shift from the strategic bombing campaign to the transportation plan — the intensive targeting of French railways, coastal gun positions, and road bridges designed to isolate the Normandy beachhead from German reinforcements.
df_1_5_junio = df_2_operations[
(df_2_operations['Target Country'].str.upper() == 'FRANCE') &
(df_2_operations['Mission Date'] >= '1944-06-01') &
(df_2_operations['Mission Date'] <= '1944-06-05')
]
| Metric | Value |
|---|
| Total missions | 922 |
| Mean tonnage per mission | 46.3t |
| Median tonnage | 36t |
| Max tonnage | 602t |
| Top aircraft | B17 (363), GB17 (160), B24 (156) |
| Rank | City | Missions |
|---|
| 1 | BERCK SUR MER | 72 |
| 2 | WIMEREUX | 70 |
| 3 | BOULOGNE SUR GESSE | 67 |
| 4 | EQUIHEN | 63 |
| 5 | HARDELOT | 48 |
adi_master_1_5_junio.csv and feeds the “1–5 June 1944” dashboard view in Power BI.
Derived Columns
A.D.I. created three enrichment columns applied to adi_dataset_master.csv by parsing and normalising the raw ordnance tonnage fields from 2_operations.csv:
# Step 1: Normalise null values to 0
df_master_dia_d['High Explosives Weight (Tons)'] = (
df_master_dia_d['High Explosives Weight (Tons)'].fillna(0)
)
df_master_dia_d['Fragmentation Devices Weight (Tons)'] = (
df_master_dia_d['Fragmentation Devices Weight (Tons)'].fillna(0)
)
# Step 2: Create named aliases
df_master_dia_d['he_tons'] = df_master_dia_d['High Explosives Weight (Tons)']
df_master_dia_d['frag_tons'] = df_master_dia_d['Fragmentation Devices Weight (Tons)']
# Step 3: Classify weapon type based on which ordnance categories are present
conditions = [
(df_master_dia_d['he_tons'] > 0) & (df_master_dia_d['frag_tons'] > 0),
(df_master_dia_d['frag_tons'] > 0),
(df_master_dia_d['he_tons'] > 0),
]
tipos = [
'Ataque Combinado (HE + Fragmentación)',
'Bombardeo de Fragmentación',
'Bombardeo Pesado HE',
]
df_master_dia_d['weapon_type'] = np.select(conditions, tipos,
default='Sin Datos de Armamento')
adi_dataset_master.csv (D-Day missions, France):
weapon_type | Missions |
|---|
| Bombardeo Pesado HE | 493 |
| Ataque Combinado (HE + Fragmentación) | 33 |
| Bombardeo de Fragmentación | 9 |
weapon_type column is a critical input for the UXO risk map described in the Archaeological Findings page — missions classified as Ataque Combinado carry the highest probability of leaving intact unexploded ordnance in the soil.