Tidied Up: How We Finally Use GitHub's Cache Properly 🧹

In the last technology post we measured what our build-and-check pipeline really costs — and found: not money, but waiting time and storage. That article ended with "the clean-up work is planned but not done yet". It turned out to go faster than expected: as of midday today it's on the main branch. Here are the numbers.

The result first. Our repository's cache usage has dropped from 5.32 GB across 63 entries to 2.46 GB across 13. The C# security analysis that used to run on every pull request now takes 4 seconds there instead of 4½ minutes — while still checking everything that ships. Both without dropping a single check that protects you.

Briefly: what is a cache for? Every build starts on a freshly created, empty machine. So that it doesn't begin from zero every time, it may store intermediate results — next time it downloads them instead of recreating them. You get 10 GB per repository for that. Once it's full, GitHub evicts whatever went unused longest. And that was exactly our problem: the space was occupied, just by the wrong things.

Mistake 1: a key without a platform. Unity creates a huge intermediate folder when building. Four builds use it — Windows, Linux, macOS, WebGL — and each needs its own. The Windows build was the only one whose cache key carried no platform marker. So it matched the first available entry, which was WebGL's: 1.7 GB downloaded, which Unity then correctly recognised as "doesn't belong here" and re-imported from scratch. In the log: 257 seconds of import instead of 85. The key is now Library-Windows-, and the spook is over.

Mistake 2: nobody ever saved it. The more embarrassing find. All four builds dutifully tried to restore the intermediate folder — but not one workflow ever wrote it. The single entry that existed at all came from a tool you start by hand. So the project's most important cache was being maintained by accident. 🙈 Now it gets written on purpose.

The decision that saves space: only the critical path counts. The obvious move would have been to cache all four platforms. A look at the timings of a real release shows why that would be nonsense: all four builds start simultaneously, Windows finishes after 11 minutes, Linux and macOS after 13.7 — and WebGL after 31.6. As long as WebGL is running, the others are waiting anyway. So only its cache shortens the wait; the other three would have cost around 4.5 GB of our 10 GB budget and saved exactly zero minutes. So we cache precisely one.

Mistake 3: Docker ate the rest. When building our server images, every intermediate layer used to be stored as cache — roughly 3.27 GB across 24 entries, 64% of the entire budget, refreshed on every push. That flood reliably evicted the one Unity cache that matters. Now only the final stage is stored, and each of our four images gets its own namespace — previously they shared one and kept throwing each other out. 3.27 GB has become about 150 MB.

Security analysis: check where it counts. CodeQL, GitHub's tool for finding security holes, was our second-biggest time sink: 230 minutes across eight days, almost all of it the C# part, which rebuilds the entire project for its analysis — on every pull request. That now runs in full when merging into the main branch and once a week; pull requests keep the fast checks. Measured after the change: 4 seconds in a pull request, an unchanged good 4 minutes on the main branch. So every line that actually reaches you is still checked — just not three times over.

Small stuff that adds up. The .NET packages are cached now, but only on the path a human waits on. An emulator for foreign processor architectures is only set up when we genuinely build for one. A clean-up command that had nothing to do on a freshly booted machine, yet cost up to 1.4 minutes per build, is gone. And the intermediate files of a release build are cleared after 7 days instead of 90 — 703 of them totalling 7.4 GB had piled up.

What we deliberately did not do. Two proposals from our own analysis were dropped after measuring, rather than sold as pretty lines in a changelog. Splitting the tests across two parallel jobs would have gained exactly four seconds (one part takes 2:54, the other 4 seconds) — and would have renamed a required check, leaving every pull request stuck "waiting" forever. And skipping the big test gate before a release would also have saved nothing, because it's long finished while WebGL is still building. An optimisation that gains nothing but removes a safety net isn't an optimisation.

And then a real bug fell out of it. In the middle of this clean-up, every pull request suddenly failed a time limit: a single test took 212 seconds where 120 are allowed. The second-slowest in the same run: 10.5 seconds. The cause was an error path in our server that simply severed a broken HTTP request instead of closing it cleanly. On Linux the system only reclaims such connections in a sweep that runs every two minutes — and shutdown was patiently waiting for it. On Windows it never showed, which is why it only ever bit on the build server. Now the server answers such a case with an honest error code and closes the connection itself: the test finishes in a good 20 seconds — and real players hitting such a hiccup will get a comprehensible error instead of a severed connection. A CI clean-up day that finds a production bug is a good day. 🐛

None of this changes anything for you directly — no new block, no new animal. It simply shortens the road between "finished coding" and "installable for you". And it proves the same sentence as last time: measure first, then touch. 🚀