How Quantamentry is built

Economic data is revised after it is published. If a pipeline keeps only the newest value, a score computed today for last March uses numbers nobody had last March, and every backtest built on it flatters itself.

What we did. Every data point keeps its publication date, estimated where the source gives none, and the day we first saw it, which is never estimated. A revision never overwrites: the old row stays, marked as replaced. Every scoring step asks for the data as it stood on the day being scored.

What it caught. IMF forecasts for 2031 were stamped with this year's release date, so the newest-first lookup was reading a 2031 projection as today's value in four places. The fix pins the year on every such read, and a test now fails if a forecast row can reach a current-value read.

A nightly job can finish without error and still be wrong: a source publishes nothing, a series quietly stops, a statistics office re-bases its index.

What we did. Every ingest step reports how many rows it wrote, the newest release it holds, and whether the source has stalled against its own publishing schedule. A series that has stopped is refused rather than carried forward. Price indices are checked for re-basing before any ratio is taken across them.

What it caught. Three sources wrote nothing for six weeks inside runs that stayed green, and nothing could say whether that was a fault or a quiet source. Splicing two price indices with different base years once published inflation of −89.2% for Türkiye; the true figure was 32.1%. On the day the inflation pages shipped, the same guard refused Italy, whose index had been re-based four months earlier.

A composite score has to do something when a country lacks one of its inputs. The easy answer, re-weighting over what is present, rewards absence.

What we did. A missing dimension now counts as a neutral 50 at its full weight. A country needs enough measured dimensions to appear in the headline ranking at all, and the page says which ones it lacks.

What it caught. Under the old rule the countries with the least data ranked highest: three countries with no policy rate and no central-bank statements sat above Sweden. The ranking test that now guards this compares a sparse country with a fully measured peer.

Free data is not always free to republish. Some of the most useful series here may be used to compute a score but not passed through to a reader.

What we did. The list of restricted sources is a filter inside the SQL of every public route. Tests fail the build if a response model gains a field that would leak a restricted number, even under a new name. Where an open-licence source carries the same statistic, the pipeline uses it, which is what made the public inflation pages possible.

What it caught. The inflation calculator reads one source only, the one whose licence allows republishing its index. The test checks the bound query parameter, not a comment.

Central-bank statements are text, and a language model reads them well. It also gives a different answer each time, and will narrate a number it made up as fluently as a real one.

What we did. Each distinct statement is scored once, at temperature zero, and the published stance is the median of its scores. The weekly briefing is written by a model, then checked: every number and every country in the prose must trace back to the data it was given. If the check fails, a plain template goes out instead.

What it caught. Before the temperature was pinned, the same text scored up to 0.85 apart on a scale of −1 to +1, and one statement moved Switzerland's headline score by 20 points. The first briefing told a true-sounding story about Hungary that was really a change in data coverage, so moves that large are now held back before the model sees them.

A scoring method that never changes is wrong for longer. One that changes silently makes every chart a lie at the seam.

What we did. The method is versioned and every score row carries the version that made it. When a change re-baselines the panel, week-over-week moves across that date are withheld everywhere, rather than shown as news. The changelog is public.

What it caught. The method is on its eighth version. One lesson along the way: the site and the email digest each computed their own weekly change, so the email announced moves the site refused to show. There is one function now.