Redesigning source control at Anthropic
Interesting piece from Anthropic describing why and how they built their own custom alternative to Git, based on the dramatic increase in rate of changes to their code after the …
“Antique” is their new code review system. The source control system is called Scion, and sounds to me like a reinvention of centralized source control along the lines of Subversion.
(Update: Anthropic credit Google’s Piper and Meta’s Sapling as “prior art” - it looks like the core concept of “derived data” was heavily inspired by Mononoke, the server component of Sapling.)
They had some pretty extreme requirements:
The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
The new system was built by a team of seven people, and the migration from Git was completed in five months.