Weather Raccoon
Dependency-Aware Anomaly Laboratory V4
A month is not strange because one number is large. It is strange when the weather variables fail to behave together the way Pennsylvania usually behaves. V4 keeps the complete feature engineering from the original design, then layers conditional, joint, temporal, spatial and era-aware rarity on top.
How far into a tail?
Record heat, record wetness, huge dry spells, violent temperature swings. Useful, but it can miss strange combinations.
Do these pieces belong together?
The main prize. A month can contain no record and still have a historically improbable configuration.
What should have happened?
Temperature given rainfall. Soil moisture given antecedent rain and heat. Rx1 given monthly total. Every variable gets judged in context.
Why did it rank?
The final system must name the relationships responsible. An unexplained 8.41 is not meteorology. It is a hostage note from a spreadsheet.
The master workflow
The four engines are one machine
Marginal rarity
How unusual is each feature by itself?
uᵢ = Fᵢ(xᵢ) ; zᵢ = Φ⁻¹(uᵢ)Empirical ranks avoid pretending precipitation and spell lengths are Gaussian.
Conditional rarity
How unusual is a feature given the other features?
Rᵢ = [xᵢ − E(Xᵢ | X₋ᵢ)] / σ(Xᵢ | X₋ᵢ)This is where “cool despite being dry” or “hot despite frequent rain” becomes visible.
Joint dependency rarity
How improbable is the entire configuration?
J = −log f(x₁,x₂,…,xₖ)This engine determines the canonical strange-month rank. The other engines do not get added as duplicate votes.
Temporal + spatial rarity
Was the state already primed for this? Was the pattern local or statewide?
P(Xₜ | Xₜ₋₁,Xₜ₋₂,…) + spatial coherenceA one-month surprise and a three-month evolving regime are different scientific objects.
Feature engineering: the original design is preserved
Before any fancy dependence model gets to wear a lab coat, the daily data must describe the month properly. V4 keeps the first design’s state, duration, intensity, fluctuation, frequency, persistence and concentration dimensions.
Temperature
- State: mean Tmax, mean Tmin, mean temperature, DTR.
- Duration: hot-day count, cold-night count, longest hot/cold spells, threshold degree-days.
- Intensity: TXx, TNn, hottest/coldest multi-day windows.
- Fluctuation: temperature SD, total variation, largest 24/48 h change, DTR anomaly.
Precipitation
- Amount: monthly/seasonal total.
- Frequency: wet-day and heavy-rain-day counts.
- Intensity: Rx1, Rx3, Rx5, SDII.
- Persistence: longest wet/dry runs and event spacing.
- Concentration: top-1/top-3 shares and optional precipitation Gini.
Land surface + radiation
- Soil state: mean/minimum layer moisture.
- Duration: days below/above soil-percentile thresholds.
- Change: drying and recharge rates.
- Compound: heat + dry soil, heavy rain + saturated soil.
- Radiation: consensus anomaly, not one vote per reanalysis.
Spatial + sequence
- Statewide median/coherence across 10 climate divisions.
- Maximum local extremity.
- Fraction of divisions beyond selected rarity thresholds.
- 30/60/90-day antecedent precipitation and thermal state.
- Prior-month soil moisture where available.
Why correlated metrics do not get extra votes
Mean Tmax, mean temperature and hot-day counts overlap. Monthly precipitation, wet days and Rx1 overlap. ERA5-Land and NLDAS soil moisture overlap. Metrics are grouped into physical dimensions first. The dependency model then learns their covariance/dependence. Adding another product must improve measurement, not magically increase that physical family’s importance.
The rank-normal transformation used by the transparent baseline
Within each calendar month or season, each feature is empirically ranked. For rank r among N years:
u = (r − 0.5)/N ; z = Φ⁻¹(u)This preserves order while putting wildly different quantities onto a common latent scale. Production models still retain the original values for interpretation.
Start finding strange months and seasons
The browser below uses a transparent rank-Gaussian + shrinkage Mahalanobis dependency baseline on ten core features. That is already dependency-aware, but it is deliberately labeled prototype: the final scientific leaderboard will use blocked out-of-sample evaluation and a flexible copula/conditional-density model as an independent second judge.
| # | Period | Strange | Extreme | Type |
|---|
—
—What is unusual, and what is unusual given everything else?
Marginal percentile is the ordinary rank. Conditional σ is the Gaussian-baseline residual after accounting for the other nine core features. Large disagreement between the two is exactly the relationship anomaly we are hunting.
Most Extreme vs Most Strange
Teaching interpretation
Relationship microscope: change the context, not just the observation
This intentionally simple July regression is retained as a teaching device. It is not the final anomaly model. Its job is to make conditional expectation tangible.
Each dot is one complete Lower Susquehanna July. The line holds precipitation near its median and varies wet-day count. The actual pairwise relationship is weak, a useful warning that real weather needs the multivariate engine.
Two reference frames, plus richer eras
Fixed historical rarity
Everyone competes against a common yardstick. This is the permanent 1893-present Core and lets 1901 legitimately compete with 2025.
Core = same required variables + same feature definitions across all yearsContemporaneous / era rarity
Answers a different question: how surprising was this month for the climate and observing era of its own time? Both scores are retained because climate change otherwise turns one question into two and then quietly pretends nothing happened.
Era rarity = rank within the appropriate historical information/climate regimeCore temperature + precipitation. Snow supplementary.
Add ERA5-Land radiation and soil fields.
Add NARR + NLDAS; MERRA begins 1980.
Add POWER; mature multi-reanalysis overlap.
Add HRRR as the high-resolution modern layer.
Seasons are recomputed from daily data
Season scores are not the average of three monthly scores. V4 rebuilds state, duration, intensity, volatility, precipitation persistence and concentration across the entire continuous season, then evaluates the resulting seasonal feature vector.
Integrated season
All daily observations are treated as one continuous physical episode.
Peak shock
The production report separately records the most anomalous constituent month/event so one spectacular episode is not averaged into invisibility.
Winter continuity
December + January + February are a continuous winter. For example, December 1935 through February 1936 is Winter 1935–36.
Weather phenotypes: rank it, then name what kind of strange it was
Persistent heat-drought
High temperature state/duration plus low precipitation and long dry persistence.
Flashy concentrated wet
High total/intensity but comparatively few wet days and a large fraction falling in one or three events.
Volatile transition
Large day-to-day temperature variability or abrupt thermal jumps, even when the monthly mean is ordinary.
Cold-wet / warm-night / dry persistence
Other recurrent physical signatures. Production clustering will discover data-driven families rather than relying only on these heuristic labels.
From transparent baseline to defensible scientific ranking
Judge A · transparent baseline
Rank-Gaussian features + regularized covariance/precision matrix + Mahalanobis distance.
D² = zᵀ Σ⁻¹ zFast, interpretable and an excellent sanity check. The interactive browser uses this family of model.
Judge B · flexible dependency model
Vine copula or comparable conditional-density model preserving non-linear/tail dependence.
f(x) = c(u₁,…,uₖ) · ∏ fᵢ(xᵢ)The final strange-month rank comes from out-of-sample joint rarity, with disagreement between judges reported as uncertainty rather than hidden.
Production safeguards
Why Mahalanobis remains even after the vine copula exists
Because sophisticated models can be confidently wrong. A simple covariance baseline is an independent diagnostic. If the vine declares a month historically impossible while the transparent baseline shrugs, that disagreement deserves investigation, not ceremonial burial.
How the final “why” should be generated
The production explanation should decompose out-of-sample log-surprise with a Shapley-style attribution over feature families/relationships, then pair that with conditional residual diagnostics. The text should state relationships, not merely feature values: “Tmax was much warmer than expected given 18 wet days and high soil moisture,” for example.
What one finished record should eventually look like
July 19XX · Lower Susquehanna
Why: The month was not record-hot and not record-wet. Its rarity came from sustained high daytime temperature occurring alongside unusually frequent precipitation, moist antecedent conditions and suppressed radiation, a combination rarely observed together.
Actual uploaded data inventory
The permanent core truly spans the record, while later environmental fields arrive in layers. That is why V4 separates the timeless Core from era-enhanced interpretation.
| Family | Variable | Start | End | Non-missing daily rows |
|---|---|---|---|---|
| Permanent core | Daily Tmax | 1893-01-01 | 2026-09-10 | 913,986 |
| Permanent core | Daily Tmin | 1893-01-01 | 2026-09-10 | 913,986 |
| Permanent core | Daily precipitation | 1893-01-01 | 2026-09-10 | 913,986 |
| Historical supplement | Snowfall | 1893-01-01 | 2026-09-10 | 873,939 |
| Historical supplement | Snow depth | 1893-01-01 | 2026-09-10 | 817,550 |
| 1950+ land surface | ERA5-Land solar radiation | 1950-01-01 | 2026-09-10 | 532,219 |
| 1950+ land surface | ERA5-Land 1 m soil moisture | 1950-01-01 | 2026-09-04 | 532,114 |
| 1950+ land surface | ERA5-Land deep soil moisture | 1950-01-01 | 2026-09-04 | 532,114 |
| 1979+ land surface | NARR solar radiation | 1979-01-01 | 2026-08-31 | 330,790 |
| 1979+ land surface | NLDAS 1 m soil moisture | 1979-01-02 | 2026-09-05 | 313,452 |
| 1979+ land surface | NLDAS deep soil moisture | 1979-01-02 | 2026-09-05 | 313,452 |
| 1980+ radiation | MERRA solar radiation | 1980-01-01 | 2026-07-31 | 322,725 |
| 1984+ radiation | POWER solar radiation | 1984-01-01 | 2026-09-07 | 296,216 |
| 2013+ high resolution | HRRR solar radiation | 2013-08-15 | 2026-09-10 | 85,338 |
All 19 Pennsylvania series in the uploaded file
| ID | Name |
|---|---|
| PAC001 | Pennsylvania - Pocono Mountains Climate Division |
| PAC002 | Pennsylvania - East Central Mountains Climate Division |
| PAC003 | Pennsylvania - Southeastern Piedmont Climate Division |
| PAC004 | Pennsylvania - Lower Susquehanna Climate Division |
| PAC005 | Pennsylvania - Middle Susquehanna Climate Division |
| PAC006 | Pennsylvania - Upper Susquehanna Climate Division |
| PAC007 | Pennsylvania - Central Mountains Climate Division |
| PAC008 | Pennsylvania - South Central Mountains Climate Division |
| PAC009 | Pennsylvania - Southwest Plateau Climate Division |
| PAC010 | Pennsylvania - Northwest Plateau Climate Division |
| PATABE | Allentown Area |
| PATAVP | Avoca Area |
| PATERI | Erie Area |
| PATIPT | Williamsport Area |
| PATMDT | Middletown-Harrisburg Area |
| PATMPO | Mount Pocono Area |
| PATPHL | Philadelphia Area |
| PATPIT | Pittsburgh Area |
| PATRDG | Reading Area |