Technical overview

ScaleGres: real PostgreSQL, with compute separated from storage

Every project keeps its own unmodified PostgreSQL 18 instance. Durability moves from a per-VM disk to a shared, batched commit path on object storage (S3), so instances become disposable and many of them can share a handful of hosts.

–
compute hosts needed (RAM-bound)
–
fewer machines than one VM per running project
–
less RAM to buy
–
less provisioned storage

Capacity model: edit any value, the table updates instantly

Left: starting assumptions (all editable). Right: resources used by a one-VM-per-project fleet versus ScaleGres shared cells for the same projects and the same workload. Changes are also encoded in the page address, so a configured scenario can be shared as a link.

Fleet
Today: one VM per project
ScaleGres cells
Compute host
Data
ResourceOne VM per project (today)ScaleGres shared cellsDifference

What ScaleGres is

ScaleGres turns PostgreSQL into a compute layer on top of object storage. A project is a normal PostgreSQL instance (a cell) whose write-ahead log (WAL) is made durable in S3 instead of on a local disk. Because everything that must survive lives in S3, a cell can be stopped, moved or recreated on any host in seconds, and idle projects need no running compute at all.

Today: one VM, one disk, per project

  • Each project owns a VM with its own OS, memory reservation and block volume.
  • Memory and vCPU are sized for the peak and sit mostly idle.
  • The volume is provisioned (and paid) even when the project is paused.
  • A host-level failure means restoring a volume.

ScaleGres: disposable cells on shared hosts

  • Dozens of cells share a host; each is an isolated PostgreSQL process tree with its own data directory.
  • Durable state is the WAL stream plus periodic base generations in S3; local disk is only a cache.
  • Idle projects scale to zero; a cell restores from S3 on the first connection.
  • Losing a host loses nothing: its cells restart elsewhere from S3.

Architecture

ScaleGres architecture Clients connect through a gateway to PostgreSQL cells on compute hosts. Cells send WAL to a durability agent that batches commits from many tenants into shared objects in S3. A control plane places and wakes cells. Clients (any PostgreSQL driver) Gateway / router routes by project, wakes cells Compute host (shared by many cells) Cell APostgreSQL 18 Cell BPostgreSQL 18 Cell CPostgreSQL 18 Cell DCell E idle: no cell local NVMe = cache only Durability agent (HA) batches commits of many tenants into one object per window S3 object storage shared immutable WAL batches + content-addressed base generations Catalog busDDL epochs for read replicas Restore pathgeneration + WAL tail → cell Control plane placement · scale-to-zero wake-on-connect · host recovery periodic generations state itself lives in S3 WAL flush PUT start / stop

Compute cell

A complete PostgreSQL 18 server for one project: its own postmaster, shared buffers, extensions and data directory. Only the point where WAL becomes durable is redirected to the durability path.

Durability agent

Receives WAL ranges from many cells and writes them as one immutable, checksummed object per short time window. The commit of every tenant in the window is acknowledged by that single shared PUT.

Object storage

S3 holds the shared WAL batches and periodic base generations (content-addressed chunks, so unchanged data is stored once). Together they are enough to rebuild any cell.

The commit path

  1. A backend commits. PostgreSQL's WAL writer is about to mark a WAL range as flushed.
  2. A small hook, called just before the flush pointer advances, sends that range to the durability agent.
  3. The agent adds the range to the current batch together with ranges from other tenants and writes the batch to S3 with a conditional, create-if-absent PUT (shared commit).
  4. When the object is durable, the agent acknowledges every request in it; PostgreSQL then advances its flush pointer and reports COMMIT to the client.
  5. Retries are idempotent: a range is identified by tenant, timeline and LSN, and re-sent bytes are verified against what is already stored.

Batching is what makes object storage practical for commits: with many tenants sharing each window, a single PUT carries hundreds of commits, so the request rate to S3 stays small even when the number of projects is large.

Disposable compute and scale to zero

Why a fleet needs far fewer hosts

1. No VM per project

The OS, kernel and reserved-but-unused memory of tens of thousands of VMs disappear; many cells share one host's memory and CPU.

2. Idle means off

Only projects that are active right now hold a cell. The rest cost storage in S3 and nothing else.

3. Storage on usage

Data is paid for as stored in S3, not as provisioned per volume, and it is shared across restarts and hosts.

PostgreSQL compatibility: it is PostgreSQL

ScaleGres does not reimplement PostgreSQL and does not sit behind a protocol proxy that emulates it. The server is stock PostgreSQL 18, changed in a minimal way by a small patch; everything else is done outside the server.

The patch

  • About 40 lines in two files (xlog.c, xlog.h).
  • A durability hook called in XLogWrite() before the flush pointer advances.
  • An optional switch to skip the redundant local fsync when durability is already provided by the hook.
  • Both are off by default; with them off the binary behaves exactly like upstream. The upstream regression suite passes with the patch applied and with durability enabled.

Shared-commit object storage

  • Durability uses S3 with a shared commit: commits of many tenants are combined into shared, immutable objects, in the spirit of Turso's approach to S3-backed commits.
  • Conditional (create-if-absent) writes give atomic publication; checksums protect every object.

What that means for users and tooling

  • 100% SQL compatibility: same parser, planner, executor, data types, MVCC, transactions, isolation levels and error behaviour, because it is the same code.
  • 100% extension compatibility: extensions load into the unmodified server, including C extensions such as pgvector, with no changes and no per-extension adaptation, because the server’s extension interface is untouched.
  • Same wire protocol and drivers: every PostgreSQL client, ORM and BI tool works unchanged.
  • Same on-disk format and WAL format: base backups, pg_dump/pg_restore, pg_basebackup and standbys all use the formats they already know.
  • No migration of application code: no new query dialect, no unsupported statements, no behavioural surprises.
  • Upgrade path: new PostgreSQL releases need only the small patch re-applied.

Other advantages

Starting assumptions

The defaults of the model above. They are estimates built from public information and general experience, not telemetry from any platform, and every one can be changed in the form.

AssumptionDefaultBasis
Fleet90,000 Free + 10,000 ProRound numbers for a large managed fleet. Pro projects are always running and never scale to zero.
Free VM today0.5 GB, 2 shared vCPUPublic plan description: Free has 500 MB RAM and shared CPU.
Free projects holding a VM today30%Free projects are paused after 7 days of inactivity (public), so many idle projects still hold a VM.
Pro VM size mix70% 1 GB, 20% 2 GB, 8% 4 GB, 2% 8 GB (average 1.58 GB)Public default is the 1 GB "Micro" size; larger sizes are paid add-ons; the split is an estimate.
Provisioned volume today8 GB per projectVolume paid for even when the project is paused.
Free projects with a cell up at a given moment6%Hobby and development databases are idle almost all the time; cells stop after a few idle minutes.
Memory per active Free cell0.30 GBAn idle PostgreSQL cell with small shared buffers measured around 70 MB before client sessions and page cache; plus a few backends and a small working set.
Average CPU per cell0.03 vCPU Free, 0.10 vCPU ProTypical low average utilisation with bursts.
Pro cell memory vs. its VM90%VM and OS overhead and unused headroom disappear; the working set stays.
Host64 GB / 8 vCPU, 85% RAM, 60% CPUMemory-optimised instance class; utilisation targets leave room for page cache and bursts.
Data per project50 MB Free, 3 GB ProTypical sizes; object-store amplification 2× (base generations and WAL).
Not modelled–Network, backups, read replicas, high-availability spares of shared services (the spare-hosts field adds a percentage for host failures).

How hosts are computed: hosts needed = the larger of (RAM needed ÷ usable RAM per host) and (average CPU demand ÷ usable vCPU per host), then rounded up and increased by the spare percentage.