Ship fast by building platforms

August 24, 2024

There is a way of writing software that I have seen a couple times that I would like to call out. I don’t know if there’s a name for it, but here is the situation:

• You need to ship a lot of widgets to production
• Each widget is just different enough from each other that a naive approach would necessitate building each one individually

For example, if you need to deploy a ton of language models to a production API, some may be in safetensors format, others might be bin. Each has their own way of interfacing. Other considerations may be:

1. Building the model
2. Data collection
3. Data curation
4. Request routing

The right thing to do in this case is to build one or two bespoke implementations, then once obvious patterns emerge, build a platform as quickly as possible. Yes, during this time you won’t be able to ship widgets as quickly (sometimes not at all), but it has been my experience that once a platform is built widgets can be shipped thousands of percentage points faster. No joke, in one case there was one model that took 6 months to ship. After the platform it took 6 hours.

This doesn’t just make it easier to ship, it gives you and your team bragging points about how fast you can ship. The bad part is that if people don’t recognize the platform as a more general solution and view the initial implementation as a one off, it can be difficult (politically speaking) to pitch to management that the whole team should divert engineering resources to building another platform.

CI/CD is a type of platform. The bespoke implementations are the commits. After a couple commits and some bespoke setting up of servers, you find out what needs to be done, and you build a platform for it that accepts commits of nearly any kind and produces a shipped product as a result.

If you want to ship fast, take the time to build a platform. You will thank yourself later.