The day the IMF data host stopped resolving

Quantamentry — migrating to a new SDMX endpoint, and the silent-staleness trap we found on the way.

Every data pipeline eventually inherits a source that just... moves. This is the story of one that did, what it took to follow it, and the one gotcha in the migration that is worth stealing whether or not you ever touch IMF data.

The symptom

Monthly CPI for 27 of our non-OECD countries went quietly stale. The most recent observation for each of them was frozen at December 2025, and it stayed frozen while every other source kept ticking. Nothing in the logs screamed — the daily job ran green, the downstream scores kept computing on the last-known values, and the freshness dashboard was the only place the rot showed up.

The root cause was blunt: the legacy IMF data host, dataservices.imf.org, had stopped resolving in DNS entirely. Not a 500, not a timeout mid-request — the hostname itself no longer resolved. The IMF had migrated its data platform, and the old host was on its way out. That is the whole story; there is no villain here. Platforms get replaced, old endpoints die, and it is on the consumer to follow. The interesting part is what following it involved.

What made the symptom slippery is that this particular series was never load-bearing in a way that trips an alarm. Monthly CPI for these 27 countries flows into the credibility scoring through a fallback chain — if the freshest monthly print is unavailable, the model reaches for the next-best source rather than failing outright. That design is a feature: one dead source should not take down scoring for a whole tier of countries. But it has a cost, which is that a source can die quietly and the system keeps producing plausible numbers on slightly stale inputs. Graceful degradation and silent staleness are the same mechanism viewed from two angles, and the freshness dashboard is what keeps the second angle visible.

The migration

The new home is api.imf.org, on SDMX 2.1. The good news: it still speaks the same StructureSpecific XML dialect, so the parser we already had for the old host needed almost no change. The pain was entirely in the keys — the dataflow identifiers and dimension codes that address a series.

Two things bit us:

CPI now lives on the IMF.STA,CPI dataflow, and the all-items series is keyed with COICOP _T — the SDMX "total" convention — not the CP00 code the old platform used. Send CP00 and you do not get an error; you get nothing (more on that below). We had to remap the COICOP dimension across every CPI series.
Balance-of-payments moved to IMF.STA,BOP, and here the legacy indicator codes do not port one-to-one. The old BOP indicator identifiers had no clean mechanical translation to the new dataflow's code list, so this was not a find-and-replace — it was re-deriving each series we needed against the new structure definition and re-testing.

None of this is hard in the sense of clever. It is hard in the sense of tedious and easy to get subtly wrong, because a wrong key on this platform fails in the most dangerous way possible.

The gotcha worth stealing

Here is the part that generalises far beyond the IMF: a bad key returns HTTP 200 with an empty DataSet.

Not a 404. Not a 400. A cheerful 200 OK, a well-formed SDMX document, and zero observations inside it. If your ingestion code trusts the status line — if resp.status_code == 200: parse_and_store() — then a typo in a dimension code, or an un-migrated series key, sails straight through as "success" and writes nothing. Your job goes green. Your data goes stale. You find out weeks later from a freshness alert, if you have one, or from a customer, if you do not. This is the single most insidious failure mode in data engineering: the silent success.

Our fix is a canary. For CPI ingestion we hardcode a series we know must always return data — the United States all-items CPI — and after each run we assert that the canary came back non-empty. If the USA canary parses to zero observations, the key contract is broken somewhere and we raise rather than return an empty list. A structural break in the API can no longer masquerade as a quiet, successful, empty ingest. It is three lines of code and it is the difference between finding a migration break in one run versus one month.

The payoff

Once the keys were remapped and the canary was in place:

CPI is current to June 2026 again for the 27 non-OECD countries that had been frozen at December 2025.
BOP coverage jumped from roughly 70 to 148 countries — the new dataflow simply exposes far more of the panel than the endpoint we were on.
Reserves are now available quarterly through the same BOP dataflow, tightening a series that had been coarser before.

So the migration was not just damage control; following the source to its new home materially widened coverage. That is often the hidden upside of a forced migration — the new platform is usually the one the publisher is actually investing in. The endpoint you are reluctantly forced onto is frequently richer than the one you were comfortable with, because the publisher's own attention has moved there. It is worth remembering that the next time a deprecation notice lands and the temptation is to treat it purely as a cost.

The other quiet payoff was the canary itself. Having built it for CPI, we now have a template we apply to every SDMX-style source that can return a well-formed empty response: pick one series that must always exist, assert it comes back populated, and let a broken key contract throw instead of writing nothing. It cost almost nothing to generalise, and it turns a class of silent failures into loud ones across the whole ingestion layer.

Lessons

Two, and they are the reusable core of the whole episode:

Assert data presence, not status codes. 200 OK means the HTTP conversation succeeded. It says nothing about whether the payload contains what you asked for. For any source that can return a well-formed empty response — and SDMX platforms very much can — add a canary or a minimum-row assertion and fail loudly when it trips. Trust the data, not the status line.
Connection outages and silent-empty responses are different failure modes, and they need different handling. A DNS failure or a connection reset is loud — it throws, you retry with backoff, and if it persists you know something is down. A 200-plus-empty is quiet — no exception, no retry, no signal — and it needs a completely separate defence: an affirmative presence check. Conflating the two ("just wrap it in try/except and retry") leaves the quiet failure completely uncovered. Handle both, separately, on purpose.

The macro-data ecosystem is a moving target — hosts migrate, dataflows get renamed, dimension codes change under you. The defensive posture that makes it survivable is not clever; it is just refusing to let "the request didn't crash" stand in for "we got the data."

To see the CPI, BOP, and reserves series this migration restored — country by country, snapshot-dated — Create a free account — watch 3 countries, or read how each indicator is sourced and caveated on the methodology page.

— Quantamentry, July 24, 2026