Napa powers Google’s data warehouse needs for critical clients like Ads and payments. These clients have differing requirements around cost, performance, and data freshness, including a strong expectation of variance-free, robust query performance. At its core, Napa’s principal technologies for robust query performance include the aggressive use of materialized views, which are maintained consistently as new data is ingested across... Read More

rqlite is a lightweight, distributed database which uses SQLite as its database engine. This presentation will discuss its goals, design, and implementation, with particular reference to its use of the Raft consensus algorithm, and its embedding of SQLite. We will also discuss rqlite testing, performance, lessons learned during development, and some of its real-world applications. This talk is part of... Read More

Database management systems (DBMSs) are an important part of modern data-driven applications. However, they are notoriously difficult to deploy and administer because they have many aspects that one can change that affect their performance, including database physical design and system configuration. There are existing methods that recommend how to change these aspects of databases for an application. But most of... Read More

Code generation is a widely-used technique for improving query execution throughput by compiling instructions into native code. This technique, however, leads to design challenges for the recovery system of a DBMS. The log replay process will be disconnected from the built-in execution engine that has been modified to operate efficiently on compiled code. This usually leads to the implementation of... Read More

The rapid democratization of data has placed its access and analysis in the hands of the entire population. While the advances in rapid and large-scale data processing continue to reduce runtimes and costs, the interfaces and tools for end-users to interact with, and work with, data is still lacking. It is still too difficult to translate a user’s data needs... Read More

In this talk, I'll talk at a high level about how the PostgreSQL query planner approaches join planning, and how it gathers and uses statistics. Without losing sight of the fact that these algorithms generally work, I want to highlight some of the annoying cases where they break down, and the problems that they can cause for users and developers.... Read More

MonetDB is the pioneering open-source main-memory oriented column store developed in a research setting and spinning out into the enterprise market to make a (performance) difference. MonetDB innovates at all layers of a DBMS, e.g. a storage model based on vertical fragmentation, a modern CPU-tuned query execution architecture, automatic and self-tuning indexes, run-time query optimization, and modular software architecture. In... Read More

Modern web applications require fast reads of query results over user data. In practice, they use a complex, brittle, and tricky-to-manage caching layers to achieve this performance. In this talk, I will discuss how we built a new database system, Noria, from the ground up around the paradigm of materialized view maintenance via incremental streaming dataflow. Noria combines eager and... Read More

I'll talk about the design of InfluxDB IOx, the future core of InfluxDB, an open source time series database. It's an in-memory columnar database that uses object storage for persistence. It's written in Rust and is built on top of Apache Arrow. Unlike previous versions of InfluxDB, IOx supports standards compliant SQL and the Postgres dialect in particular. This is... Read More

Persistent memory is a new silicon technology, adding a distinct storage tier of performance, capacity, and price between DRAM and Flash. The persistent memory is physically present on the memory bus of the storage server resulting in reads at memory speed, much faster than flash. Writes are persistent, surviving power cycles, unlike DRAM. Oracle has engineered Exadata Smart PMEM Cache... Read More