Cloud cost optimisation as an engineering discipline

Most cost reduction exercises follow the same pattern. A bill crosses a threshold, someone is asked to cut it, a few idle instances are deleted, a commitment is purchased, and the number drifts back up over the following months. The exercise fails because it treats cost as an invoice rather than as a property of the system.

Our position is that cost belongs in the same category as latency and availability. It has a budget, it is measured continuously, it is attributed to the component that produces it, and it is considered when the design is drawn rather than after the system is running.

A total monthly figure cannot tell you whether a system is efficient. A bill that doubles while traffic triples is a good result; a flat bill on falling usage is a bad one. The useful number is cost per unit of work: per thousand requests, per active tenant, per processed transaction, per gigabyte ingested.

Choosing the unit is an engineering decision, because it has to follow the thing that drives load. Once it exists, it behaves like any other service-level indicator. It can be graphed against deployments, it can regress, and a regression can be traced to a change. A feature that adds a synchronous call to a managed service on every request shows up as a step in cost per request on the day it ships, which is when it is cheapest to question.

Nothing can be optimised until it has an owner. Our default is that every resource carries tags for owning team, service, environment and cost centre, and that the tags are applied by the infrastructure-as-code modules rather than by convention. Policy in the pipeline rejects untagged resources, because tagging discipline that relies on memory decays within weeks.

Shared platforms need more than tags. A Kubernetes cluster or a shared database appears as one line on the bill, so its cost has to be allocated by namespace requests, query time or storage consumed. The allocation will be imperfect. An approximately fair model that teams can see is far more useful than a precise one that arrives too late to act on.

The order matters. Committing to a year or three of capacity for a fleet that is twice the size it should be locks the waste in. We right-size from observed utilisation over a representative period, looking at high percentiles of CPU and memory rather than averages, and at the resource that actually saturates. A service limited by memory gains nothing from a larger CPU allocation.

Commitments such as reserved capacity and savings plans are then a financial instrument applied to the stable baseline, the load that exists at the quietest hour of the week. Covering peak with commitments means paying for idle hours. Interruptible capacity suits work that tolerates being stopped: batch jobs, CI runners, stateless workers behind a queue. It does not suit a primary database or anything whose recovery time is longer than the interruption notice.

Compute is the visible part of the bill and often the easiest to reason about. The line items that surprise teams are created by design decisions, and no amount of instance tuning removes them.

Autoscaling reduces cost only when it scales on a signal that tracks demand and scales down as readily as it scales up. CPU is a poor signal for a worker that waits on I/O; queue depth or in-flight requests per instance is usually closer to the truth. Slow start-up times force larger buffers, so reducing image size and warm-up work is cost work.

On Kubernetes, the scheduler places pods by their resource requests, not their usage. Inflated requests produce nodes that look full and sit mostly idle. Reviewing requests against actual usage often recovers more capacity than any change to the node pool. Non-production environments that scale to zero outside working hours are the simplest saving available, provided they can be recreated reliably from code.

Cost efficiency competes with other properties, and it should sometimes lose. Headroom is what absorbs a traffic spike or the loss of a zone, so a system running at very high utilisation has traded resilience for a smaller bill. That trade should be explicit and tested, not an accident of an aggressive scaling policy.

Engineering time is also a cost. For an early product with a small bill and an unproven market, a week spent shaving infrastructure spend is usually a week misallocated, and managed services that cost more per unit are the correct choice because they remove operational work. The discipline we argue for is measurement and ownership from the start, with optimisation effort applied where the unit cost shows it will pay back.