# The oracle you build yourself > Dependabot opened five PRs against this site, three of them major. There is no test suite here, so the check had to be built: render the site twice, once with the old renderer and once with the new one, and diff what came out. Source: https://rfriedmann.de/blog/the-oracle-you-build-yourself/ Published: 2026-08-11 · Track: log · Level: Ops Dependabot opened five pull requests against this site. Three were major bumps: `node` 22 → 26 in the build stage, `nginx` 1.27 → 1.31 in the serve stage, and [Astro](https://astro.build) 6.4.7 → 7, the renderer that turns the markdown you're reading into HTML. This repository has no tests. It has no CI. And a merge to `main` is a deploy: the workflow SSHes into the box, resets to `origin/main`, and rebuilds the container that serves rfriedmann.de. So the question was the same one I asked when [bumping a dependency I couldn't read](/blog/bumping-a-dependency-you-cant-read/), except this time none of the machinery that answered it existed.
5 → 1
PRs, merged as one commit (the two npm PRs collide on the lockfile)
3
of them major version bumps
1483
text blocks compared between the old and new render
2
real differences, one of which was a fix
## The failure mode isn't a crash A renderer that breaks loudly is the easy case. The build exits non-zero, the image never gets built, the old container keeps serving, and you find out in ninety seconds. What I was actually afraid of is the quiet version. A major version of a renderer changes how markdown becomes HTML and how that HTML gets minified. The build stays green, forty-five pages come out the other end, and some of them are *slightly* different: a heading that lost its anchor id, a paragraph where two words ran together, an entity escaped twice so a reader sees `<` in the middle of a sentence. Nothing errors. It just quietly ships. Reading the upgrade guide doesn't settle that. An upgrade guide tells you what the authors thought was worth mentioning, and it's written against every site, not mine. It can't tell me whether *this* markdown, with *these* forty-five pages, still comes out the same. That's not a documentation problem, it's a scope problem: only my content can answer a question about my content. ## A static site is a pure function Here's what makes this tractable, and it's the whole trick. A static site generator is a pure function. Feed it a git tree, get a directory of files. Same input, same output, every time. So I don't need a test suite. I need the same input run through both compilers: ```bash # the new build, from the working tree with all five bumps applied docker build -t rfriedmann-new . # the old build, from an untouched checkout of main git worktree add "$SCRATCH/old" main docker build -t rfriedmann-old "$SCRATCH/old" # pull dist out of both images for v in old new; do cid=$(docker create "rfriedmann-$v") docker cp "$cid":/usr/share/nginx/html "$SCRATCH/dist-$v" docker rm "$cid" done ``` The first comparison is free and nearly worthless. The file lists are identical apart from one content hash in a stylesheet name. Good. Then I compared the files themselves, and all forty-five HTML pages differed. That's not a signal, that's noise with a number attached. ## The oracle was wrong before the code was So I wrote something sharper: strip the tags, compare the visible text. Every page still differed, and every page differed *in the same place* — right at the top, in the favicon. ``` alt: "...Raphael FriedmannRF\"> Skip to content Raphael Friedmann The log..." neu: "...Raphael FriedmannSkip to contentRaphael FriedmannThe log..." ``` That `RF">` is not on the page. It's an artefact of my own stripper. The favicon is an inline SVG in a `data:` URI, and Astro 6 emitted its angle brackets raw inside the attribute, so my `<[^>]*>` regex hit a `>` that belonged to the *value* and thought the tag had ended. Astro 7 escapes them as `<`, so the same regex swallows the whole tag cleanly. Two different renderings, identical to a browser — an HTML parser decodes the entity right back — and my sieve reported it as a difference on every single page. The first thing a homemade oracle finds is usually a bug in the homemade oracle. That is not a reason to skip building one; it's a reason to look at what it says instead of trusting the count. A checker that reports "45 of 45 pages changed" has told you nothing, and the temptation at that moment is to wave it off as whitespace noise and merge. That's precisely when it's worth another twenty minutes. ## Asking a question worth answering The insight that made the difference: whitespace *between* block elements is invisible — the layout decides that spacing, not the markup — while whitespace *inside* a sentence is the whole ballgame. So compare only the text inside prose elements, and classify each difference: ```js // same block, both builds. If removing every space makes them equal, // only whitespace moved. If not, actual characters changed. const strip = s => s.replace(/\s/g, ''); if (a === b) continue; // identical if (strip(a) === strip(b)) { wsOnly++; } // whitespace only else { real++; report(a, b); } // this is the interesting pile ``` Forty lines of JavaScript, most of it walking directories. And now the answer means something:
What survived each stage of the sieve
[diagram omitted — see the page for the chart]
Every paragraph, list item, heading, table cell and blockquote across 45 pages, rendered by Astro 6 and Astro 7. Forty-one of the differences were whitespace. Two changed actual characters. One of those two made the site better.
## The two that mattered The first real difference was a non-event, and I want to name it anyway, because "the oracle fired twice" sounds worse than it was: ``` alt: "...divide the accumulator by the normaliser, O <- O / l, to get..." neu: "...divide the accumulator by the normaliser, O <- O / l, to get..." ``` `<` and `<` are two spellings of `<`. Different bytes, same character, same page. The second one is why I'm writing this post. A heading in [the roofline piece](/blog/prefill-vs-decode-roofline/) is written in markdown as `## "Compute is free"` — straight quotes, which the renderer turns into typographic ones. Here's what each version produced: ``` alt: id="compute-is-free">”Compute is free” U+201D … U+201D neu: id="compute-is-free">“Compute is free” U+201C … U+201D ``` Astro 6 opened that heading with a *closing* quotation mark. Astro 7 opens it with an opening one. That's a typographic bug that has been live on this site since June, on a page I have re-read a dozen times, and I never once saw it. The upgrade I was scrutinising for regressions quietly fixed something instead.

What a green build proved

What the diff proved

## The forty-one that didn't matter, and why Astro 7 compresses HTML with JSX whitespace rules, which is more aggressive than what came before. Mostly that lands between flex items, where it's invisible by construction. One case needed a real look: ``` alt:

27 March 2026 … neu:

27 March 2026… ``` That space is gone, and `.post-meta` is a plain block, not a flex container — so nothing supplies the gap automatically. Whether this is a regression or a rounding error lives entirely in the stylesheet, and the stylesheet says the badge is `inline-flex` with `margin-left: .6rem`. So the date and the badge are about four pixels closer together than they were, and nothing collides. ## What I did not verify I argued that last paragraph from the CSS, not from a screenshot. The browser I'd have used for a visual diff had no Chromium installed, and I wasn't going to install a browser on this host or pull a two-gigabyte image to confirm four pixels. So the honest version is: the layout is verified *by construction* — every container whose whitespace changed is either flex with a `gap` or has an explicit margin — and not *by eye*. That's a weaker claim than the text comparison, which is exhaustive and mechanical. If you're doing this on something where four pixels matter, take the screenshots. ## And then the deploy failed anyway Merged, pushed, and the deploy went red inside twenty seconds. Not the bump: ``` ERROR: failed to resolve source metadata for docker.io/library/nginx:1.31-alpine: unexpected status from HEAD request: 429 Too Many Requests ``` Docker Hub rate-limited the server. Two new base images in one build — node 26 and nginx 1.31, neither cached locally — and the anonymous pull limit said no. A re-run ten minutes later went green. The failure was harmless in a way worth noticing: `docker compose up --build` failed at the *build* step, so the running container was never stopped. The site served the old, working build the entire time, and the daily news rebuild would have retried on its own regardless. But it's a clean illustration that the oracle answers exactly one question — *did the output change?* — and infrastructure flakiness is a different question, needing a different gate. Mine now retries the pull three times. ## The principle The [CUTLASS oracle](/blog/bumping-a-dependency-you-cant-read/) was expensive. It meant re-deriving 4-bit decode from the format spec and inverting a vendor library's internal swizzle, and it only pays off because it's reused on every future bump. This one took twenty minutes and forty lines of JavaScript, because the output is text and text diffs itself. That asymmetry is the point. Most projects that render something deterministic — a site, a report, a config bundle, a schema — have this oracle sitting there for the taking, and almost nobody builds it, because there's no test file to put it in and it feels like it doesn't count. The shape is identical in both cases: compare against something outside the change. For CUTLASS that was fp64 arithmetic derived from a published spec. Here it's the previous build's own output. Neither one asks whether the new code is *good* — that's not the question a dependency bump raises. Both ask whether it does something different, and then make you look at every place where it does. The last step was to stop doing it by hand. The repo now builds every pull request into a real image, starts the container, and checks that the routes it should serve come back 200 with content on them. Patch and minor updates merge themselves once that's green; majors wait for a human, because a major is exactly the case where the diff deserves reading. The mechanical part is worth automating. The judgement isn't.