Neon is a Postgres cloud service that separates compute and storage. Separation of compute and storage makes it possible to scale the components independently, and instant startup without a lengthy restore.

Amazon Aurora pioneered the separation of compute and storage for OLTP databases. Neon uses the same concept of applying the WAL stream to reconstruct database state in the storage layer. However, Neon takes that concept further, and retains the WAL in a pre-processed format, making it possible to reconstruct any version of any page, for as long as the user wants to retain the history. That enables querying the database as of an older point-in-time and making copy-on-write branches of the database in O(1) time. These capabilities make traditional backup and WAL archive setups unnecessary.

The storage system makes use of cheap cloud storage, allowing old and infrequently used data to be “swapped out” to cloud storage, and downloading it back to fast SSDs when needed again.

In this talk, I will dive into the details of how the storage system works, and how we retrofitted these capabilities into PostgreSQL.

This talk is part of the ¡Databases! – A Database Seminar Series.

Bio:
Heikki Linnakangas a long-time PostgreSQL hacker, and cofounder of Neon. He has authored many features and enhancements in PostgreSQL, mainly around indexing, storage and write-ahead logging.

More Info: https://db.cs.cmu.edu/seminar2022#db6