8 Chapter 8 — Downloading Data: Every Portal Explained
At some point, browsing the IEM’s interactive tools stops being enough. You want the numbers — a CSV you can open in Python or Excel, a time series you can run statistics on, a dataset you can join against something else. The IEM has more download pathways than almost any free weather data source, which is its strength and also its main source of confusion. Knowing which endpoint to use for a given task is not always obvious, because several of them overlap, several of them are labeled for something slightly different from what they actually do, and a few of the most powerful ones have no front-end interface at all.
This chapter provides a working taxonomy. The goal is not to document every parameter of every endpoint — the IEM’s own help pages already do that, and they are genuinely good. The goal is to give you a mental model so that when you have a specific data need, you immediately know which pathway to reach for and why.
The Four Tiers
The IEM’s download infrastructure divides into four tiers, each with a different character:
Tier 1 — UI download portals. These are web forms — select a station, choose a date range, pick your variables, click Get Data. No programming required. The output is a CSV or Excel file that downloads to your browser. Most IEM users never leave this tier.
Tier 2 — CGI/bulk services. These are the backends that power the UI portals, exposed directly as scriptable HTTP endpoints. They take the same parameters as the web forms, but via URL query strings instead of form fields. You can call them from Python, R, curl, or anything that makes HTTP requests. Output is CSV. This is where serious workflow automation starts.
Tier 3 — JSON/GeoJSON ad-hoc services. A large collection of JSON endpoints, most serving structured metadata, current conditions, or event-based data. Not all of these have UI wrappers. Many are used internally by IEM applications and have never been prominently advertised, but they all have help pages at ?help=.
Tier 4 — API v1. The formal REST API, built on FastAPI with Swagger documentation at mesonet.agron.iastate.edu/api/1/docs. Designed for smaller, fast requests — MOS data, station metadata, VTEC events, soundings. Returns JSON Table Schema responses. This is the cleanest, most stable interface, but it covers a narrower range of datasets than the CGI tier.
The IEM’s own API documentation page is refreshingly honest about the hierarchy. It notes the CGI services are “2000s era” and not amenable to sub-second response times, and it points to Tier 4 for anything that needs to run fast. That honesty is useful: if you need a year of hourly ASOS data, you want Tier 2. When you need the current MOS guidance for a station, you want Tier 4.
Tier 1: The UI Download Portals
ASOS/AWOS/METAR Download
mesonet.agron.iastate.edu/request/download.phtml
This is the portal most people mean when they say they downloaded data from the IEM. It covers the global ASOS/AWOS/METAR archive — airport weather stations across the entire world, not just the US — with hourly and special observations going back in some cases to 1900, though the continuous reliable record for most US stations starts in the early 1970s. The UI walks you through seven steps: network selection, station selection, variable selection, date range, timezone, output format, and report type filtering.
The variable list on this portal is comprehensive and worth reading once in full. The standard set is temperature in Fahrenheit or Celsius, dew point, relative humidity, wind direction and speed, altimeter and sea level pressure, 1-hour precipitation, visibility, wind gusts, sky coverage at up to four levels with cloud base heights, present weather codes, ice accretion at 1/3/6 hours, peak wind gust with direction and time (from the PK WND METAR remark), snow depth, and the raw METAR string. All of these are available simultaneously with the “All Available” option, or you can select a subset.
A few variable-specific notes that matter for real analysis:
Precipitation (p01i) is the one-hour incremental bucket reading — the amount that fell in the hour preceding the observation time. It is not a daily total. Summing p01i across 24 hours gives you a daily total, but the effective reset timing can vary slightly by station, and missing observations can create holes that are not always obvious in a simple sum. The trace precipitation convention — 0.0001 inches — is used when precipitation is reported as Trace in the original METAR.
Sky coverage (skyc1 through skyc4) uses METAR codes: CLR, FEW, SCT, BKN, OVC, VV (vertical visibility). The corresponding height fields (skyl1–skyl4) are the cloud base altitudes in feet AGL. These are the raw decoded values from the METAR remark, not derived ceiling values — if you want a ceiling field, you need to derive it yourself as the lowest BKN or OVC layer.
Present weather codes (wxcodes) are space-separated strings of METAR weather codes: RA (rain), SN (snow), FG (fog), BR (mist), TS (thunderstorm), FZRA (freezing rain), PL (ice pellets), GR (hail), and their intensity modifiers (+, -) and descriptors (MI, BC, DR, BL, SH, TS, FZ, PR). For precipitation-type analysis, these codes are the closest thing to ground truth in the archive, combining automated sensing with human observation when available.
Report type filtering (Step 6 in the UI) is worth paying attention to. The options are MADIS HFMETAR/5-Minute ASOS, Routine/Once Hourly, and Specials. For most analysis, you want both Routine and Specials — specials are the intermediate observations triggered by significant changes in conditions (a rapid temperature drop, a wind shift, the onset of precipitation). Excluding specials gives you a cleaner hourly record but loses the precise timing of significant weather events. The 5-Minute ASOS data is a separate product from MADIS and covers US stations from approximately 2000 onward; it is not available for international stations.
The aggregate request limit is 1,000 station-years. For context, that means one station for 1,000 years, or 100 stations for 10 years, or 1,000 stations for one year. This is a generous limit but it is enforced, so queries asking for many stations over long periods will fail if they exceed it.
ASOS 1-Minute Download
mesonet.agron.iastate.edu/request/asos/1min.phtml
Separate from the standard ASOS portal, the 1-minute data archive covers US ASOS stations from approximately 2000 to present. At this resolution, you can see the actual minute-by-minute wind, temperature, and precipitation evolution within an event. For post-chase analysis, front-passage timing, or extracting the exact minute of a convective peak wind observation, this is the dataset. The Iowa AWOS 1-minute archive covers a different set of stations (Iowa AWOS only) for the period 1995–2011 and is at a separate portal at /request/awos/1min.php.
COOP / Climodat Download
mesonet.agron.iastate.edu/request/coop/fe.phtml
The Climodat download portal serves the long-period daily climate record from the NWS COOP network and the IEM’s processed Climodat stations. This is the dataset for any analysis that needs daily max/min temperature, precipitation, and snowfall going back decades. Climodat data at some stations extends to the 1880s. The download provides the IEM’s QC’d and processed daily values, not raw COOP observer submissions — station moves, missing observations, and some systematic biases have been addressed in the processing.
The companion portal for truly raw COOP observations — the daily reports as submitted by observers before IEM processing — is at /request/coop/obs-fe.phtml. If you are doing your own QC or want to understand exactly what the observer reported (including the observation time, which is essential for TOBS-aware analysis), use the raw portal. If you want the IEM’s processed daily climatology, with missing values filled where applicable and QC already applied, use the Climodat portal.
Daily Observations (IEM Computed Summaries)
mesonet.agron.iastate.edu/request/daily.phtml
Covered in Chapter 6 as a dataset — this is the UI wrapper for the IEM’s computed daily summaries across all networks. One row per station per day, with the IEM’s derived max/min temperature, precipitation, peak wind, and other variables. The network selector is the key: choose PA_ASOS for Pennsylvania airport stations, PACLIMATE for Pennsylvania COOP climate stations, and so on.
RWIS Download
mesonet.agron.iastate.edu/request/rwis/fe.phtml
Roadway Weather Information System observations — atmospheric variables plus up to four pavement temperature sensors plus subsurface temperature and pavement condition codes. Separate portals exist for the soil data at /request/rwis/soil.phtml and traffic data at /request/rwis/traffic.phtml.
DCP / HADS Download
mesonet.agron.iastate.edu/request/dcp/fe.phtml
River gauge and DCP station data, organized by SHEF variable codes. If you want river stage or discharge data from the IEM’s DCP archive rather than USGS, this is the portal. The SHEF variable selection is extensive — the HADS CGI endpoint also handles this dataset programmatically.
Other Network Portals
Additional UI download portals cover USCRN (/request/uscrn.php), Hourly Precipitation (/request/asos/hourlyprecip.phtml), and TAF forecasts (/request/taf.php). The Temperature and Winds Aloft portal at /request/tempwind_aloft.php gives you the NWS winds aloft forecast data for any station and date range.
Tier 2: CGI/Bulk Services
The CGI services are the scriptable equivalents of the UI portals, all documented at mesonet.agron.iastate.edu/api/. Each one has a ?help parameter that returns the full parameter documentation. These are the endpoints to use when you are automating a workflow, running a pipeline, or pulling data for more stations or longer periods than you want to click through in a form.
The CGI and JSON endpoints in Tiers 2 and 3 look like code, but you do not need to be a programmer to use them. A URL like:
https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py?station=MDT&year1=2023&month1=1&day1=1&year2=2023&month2=12&day2=31&data=tmpf&format=comma
is just a web address with instructions attached. Paste it directly into Chrome, Firefox, or Edge and hit Enter — your browser acts as the API client and downloads the CSV file to your desktop. No Python, no terminal, no setup required.
If you later decide to automate repeated downloads, these three lines of Python are all you need to start:
import requests
url = "https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py?station=MDT&year1=2023&month1=1&day1=1&year2=2023&month2=12&day2=31&data=tmpf&format=comma"
open("mdt_2023.csv", "wb").write(requests.get(url).content)That is the entire download. You do not need to understand it to use it — copy, paste, change the station ID and dates, run it.
The most important ones for weather hobbyists:
/cgi-bin/request/asos.py — the backend for the ASOS download portal, accepting all the same parameters as the UI form via query string. The help page at /cgi-bin/request/asos.py?help documents every parameter. A typical programmatic call for a year of Harrisburg hourly data in UTC looks like:
https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py?
station=MDT&
data=tmpf&data=dwpf&data=sknt&data=drct&data=p01i&data=vsby&
year1=2024&month1=1&day1=1&
year2=2025&month2=1&day2=1&
tz=UTC&
format=onlycomma&
latlon=no&
missing=M&
trace=T&
direct=noThe data= parameter can be repeated for multiple variables, or data=all returns everything. The format=onlycomma option returns clean CSV without the debug headers; format=comma includes headers that explain the data but are messier to parse.
/cgi-bin/request/coop.py — IEM Climodat (COOP) data programmatically. Accepts network, station, date range, and variable selection. The equivalent of the Climodat download portal as a callable endpoint.
/cgi-bin/request/daily.py — IEM computed daily summaries for any network, programmatically.
/cgi-bin/request/hads.py — HADS/DCP/SHEF data, accepting SHEF variable codes. This is the scriptable path to river gauge data.
/cgi-bin/request/hourlyprecip.py — hourly precipitation data specifically, useful when you want the precipitation record without all the other ASOS variables.
/cgi-bin/request/raob.py — RAOB sounding data programmatically, equivalent to the archive portal covered in Chapter 6.
/cgi-bin/request/gis/watchwarn.py — NWS watch/warning/advisory polygons as shapefile output, parameterized by date range, WFO, and phenomena type. The programmatic path to the warning archive that powers much of Chapter 7.
/cgi-bin/request/mos.py — MOS data as CSV for any station, model, and date range.
The full CGI service list on the API page also includes endpoints for RWIS, SCAN, USCRN, NLAE flux, Iowa soil moisture (ISUSM), SPC outlooks, SPC watches, MCDs, WPC MPDs, PIREPs, SIGMETs, G-AIRMETs, CWAs, LSRs, NEXRAD storm attributes, TAFs, temperature/winds aloft, and climate normals. Each has its own ?help page. The pattern is consistent: understand the ASOS CGI endpoint well and you understand all of them.
Tier 3: JSON/GeoJSON Ad-Hoc Services
The JSON services catalog at mesonet.agron.iastate.edu/api/ lists roughly 50 endpoints organized by category: IEM data and metadata, aviation, NWS products, NWS warnings, and miscellaneous. Unlike the CGI services, these return structured JSON or GeoJSON rather than CSV, making them appropriate for mapping applications, real-time pipelines, and programmatic workflows where you want to parse structured data rather than parse CSV.
Several of these are particularly useful for weather enthusiasts:
Current observations (/json/current.py) returns the most recent observation for any station in JSON format. The equivalent of the sortable currents table, but as machine-readable structured data.
Network GeoJSON (/geojson/network.py) returns station metadata for any IEM network as a GeoJSON FeatureCollection — station IDs, names, coordinates, and attribute flags. The fastest way to get a complete station list with coordinates for spatial analysis.
VTEC events — a family of endpoints: by point (/json/vtec_events_bypoint.py), by UGC (/json/vtec_events_byugc.py), by WFO (/json/vtec_events_bywfo.py), by state (/json/vtec_events_bystate.py). These return structured warning event data in JSON, the programmatic equivalent of the VTEC Browser.
Storm Based Warnings GeoJSON (/geojson/sbw.py) returns the actual polygon geometry for storm-based warnings for a given time window, making it straightforward to plot or analyze warning coverage spatially without downloading shapefiles.
SPC outlooks, watches, and MCDs (/json/spcoutlook.py, /json/spcwatch.py, /json/spcmcd.py) return structured JSON for SPC products — useful for building automated pipelines around convective outlook data.
RAOB data (/json/raob.py) returns sounding data in JSON format with query flexibility for specific pressure levels across date ranges, complementing the CSV portal.
Stage IV and PRISM (/json/stage4.py, /json/prism.py) return point-sample values from gridded products at specified coordinates — the programmatic equivalent of clicking on a map to get a precipitation estimate.
Tier 4: API v1
mesonet.agron.iastate.edu/api/1/docs
The formal API is built on FastAPI and exposes its full Swagger documentation at the docs URL. It is the right choice when you need fast, small responses — pulling the current MOS run for a single station, querying VTEC events for a specific county, retrieving station metadata. It returns JSON Table Schema responses with consistent structure, and the Swagger docs include a live request tester you can use directly in the browser.
The API v1 endpoints of most relevance to weather enthusiasts include MOS data (covered in Chapter 6), VTEC event data, station metadata and neighbor searches, and RAOB data. The source code for all of these lives in the iem-web-services repository on GitHub, which is worth knowing about if you want to understand exactly what a query is doing or if you find a bug and want to report it precisely.
Two Special Services Worth Knowing
MaxCSV
mesonet.agron.iastate.edu/request/maxcsv.py?help
MaxCSV is a single-variable snapshot tool — it returns the current value of one variable for all stations in a network as a CSV with station ID, latitude, longitude, and the variable value. One HTTP request, one variable, all stations simultaneously. For real-time mapping or quick tabular situational awareness, this is often faster than any other pathway. Example: current temperature for all PA ASOS stations in one call:
https://mesonet.agron.iastate.edu/request/maxcsv.py?network=PA_ASOS&var=tmpfGibson Ridge Placefiles
mesonet.agron.iastate.edu/request/grx/
If you use GR2Analyst, GRLevel3, or any other Gibson Ridge radar application, the IEM’s placefile service provides live IEM data feeds that load directly into the radar display. Placefiles are available for current ASOS observations, RWIS pavement temperatures, active warning polygons, LSRs, HADS river data, CoCoRaHS precipitation, and others. The placefile URL for each product auto-updates; you configure it once in GR2Analyst and it stays current. For real-time storm analysis overlaying IEM network data on radar imagery, this is the pathway with no programming required.
A Decision Framework
When you need data from the IEM, the question to ask is: what kind of data, at what time resolution, for how many stations, and in what format?
If you want hourly or sub-hourly observations from ASOS for one to a few stations, the UI portal at /request/download.phtml is fastest. For many stations or automation, use the CGI endpoint /cgi-bin/request/asos.py directly.
If you want daily climate data going back decades, use the Climodat portal at /request/coop/fe.phtml for processed data, or the raw COOP portal at /request/coop/obs-fe.phtml if you need observation times or unprocessed values.
If you want a derived statistic — a trend, a period average, a comparison to normals — check Autoplot first. Download the chart’s CSV export rather than doing the computation yourself.
If you want warning or NWS event data in a programmatic format, the JSON VTEC services or the CGI watchwarn endpoint are the right paths, depending on whether you need polygon geometry (JSON GeoJSON path) or tabular event metadata (CGI or API v1).
If you want gridded data sampled at a point, the IEMRE JSON API (/iemre/daily/) and the Stage IV or PRISM JSON endpoints are the fastest paths.
If you want real-time current observations in bulk, MaxCSV is the fastest single-request option. For a structured streaming workflow, iembot or the LDM feed are the appropriate tools.
What to Watch Out For
Time zone handling is not uniform across portals. The ASOS download portal lets you specify any timezone, and the timestamps in the output reflect that choice. Some JSON services return timestamps in UTC with explicit Z suffixes. Others return local standard time without a timezone indicator. Always check the timestamp format in the output before assuming UTC or local time. For any analysis that spans a DST transition, using UTC throughout and converting only at the final display step is the only safe approach.
Precipitation requires accumulation, not just download. The p01i variable in ASOS data is an incremental one-hour bucket reading, not a daily total. Gaps in the hourly record — missing observations, station outages — will silently undercount daily totals if you simply sum across 24 hours. A daily total computed from hourly ASOS data should be treated with suspicion at any station with frequent missing observations. The IEM’s daily summary portal computes these totals using its own methods, which may handle gaps differently from a naive sum.
The 1,000 station-year limit is a real constraint. A request for all US ASOS stations (roughly 1,700) for even a single year exceeds the limit. For multi-station bulk downloads, use the CGI endpoint in batches by state or network, or consider whether the daily summary portal — which aggregates to one row per station per day — reduces the data volume enough to fit within a single query.
CGI services are not versioned. The IEM’s statement on API stability is accurate but sobering: they try not to break things, but they make no guarantee. If you are building a pipeline against a CGI endpoint, build in error handling and periodic checks that the output format has not changed. The API v1 endpoints are more stable but cover a narrower range of data.
Missing data representations vary. The ASOS download portal lets you choose whether missing values are represented as M, null, or blank strings. Trace precipitation can be T, null, blank, or 0.0001 (the numeric equivalent). These choices affect how you handle missing data downstream. Set them explicitly when you configure the download, and document which convention you used.
Workflow 4 — Downloading a Year of Hourly ASOS Data
The question: You want the full 2023 hourly record for KMDT — temperature, dew point, wind speed, wind direction, and precipitation — in a CSV file you can open in Excel or analyze in Python.
Start here: https://mesonet.agron.iastate.edu/request/download.phtml
The ASOS download portal opens with a network selector at the top. From the network dropdown, select PA_ASOS (Pennsylvania ASOS). The station list on the left will populate with all Pennsylvania ASOS stations. Click MDT (Harrisburg International) to select it; it will move to the selected stations panel on the right.
Set the date range. For a full calendar year of 2023, enter January 1, 2023 as the start date and December 31, 2023 as the end date. The portal defaults to UTC timestamps; if you want observations in local time, look for the timezone selector and switch it to America/New_York (Eastern time). Be aware that ASOS stations operate on local standard time year-round, so during daylight saving time the apparent local timestamps will be one hour off civil clock time — the data is internally consistent, but the relationship to calendar dates requires care if you are computing daily totals.
In the variable selection panel, check the variables you want. For temperature and dew point: tmpf (temperature in °F) and dwpf (dew point in °F). For wind: sknt (wind speed in knots) and drct (wind direction in degrees). For precipitation: p01i (precipitation accumulation over the past hour, in inches). Check the box to include the station identifier and timestamp in the output.
Select CSV as the output format. Click the download button. For a full year at a single station, expect a file with somewhere around 8,700 rows — one per hour, with some additional rows from special observations (SPECIs) triggered by significant weather changes. This is a normal ASOS archive; do not be alarmed by rows between the standard hourly marks.
A few variables worth knowing about in the ASOS download: vsby is visibility in statute miles, wxcodes carries the present weather group from the METAR (which encodes precipitation type, fog, thunderstorm, etc.), and peak_wind_gust and peak_wind_drct capture the peak wind speed and direction within each observation period when they were recorded. If you are doing wind analysis, the peak gust columns are more useful than the instantaneous wind reading for capturing convective or frontal wind events.
The download portal has a practical rate limit: a request for more than 1,000 station-years worth of data will be rejected with an HTTP 422 response. For a single station over a single year this is irrelevant. For large bulk downloads across many stations, you will need to either split the request or use the CGI scripting interface at https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py directly.