What does it really take to bring new ideas into a production system? In this talk, I will share some experiences from my first year working on query optimization at Microsoft Azure. I will recount the story of how we discovered that Microsoft SQL Server had, in effect, been executing the Yannakakis algorithm for decades—despite its only recent resurgence in... Read More

turbopuffer is an object storage-native search engine. It puffs data into a tiered NVMe SSD and RAM cache to achieve sub-10ms warm query latency. The storage engine minimizes roundtrips to object storage to limit cold latency and maximize concurrent IOPS. It even drives consensus with object storage through conditional writes. To build a successful, independent database you need three conditions:... Read More

After decades of research, analytical database management systems (DBMSs) have become remarkably effective at optimizing and executing SQL queries. However, many users write queries that are not written entirely in SQL. Instead, these queries invoke user-defined functions (UDFs), external functions written in non-SQL programming languages such as Python or PL/SQL. UDFs provide software engineering benefits by enabling code reuse and... Read More

Chronon is a data processing framework open-sourced by Airbnb. It is adopted across organizations like Stripe, Netflix, OpenAI, and Uber. Chronon was originally built for ML applications. It has since been adopted to power a variety of use-cases—heuristics for rule engines, context for LLMs, user-facing and business-facing metrics. Chronon is adopted for its ability to generate training data at scale... Read More

Azure HorizonDB is a new PostgreSQL service that improves the OLTP performance and reliability of upstream PostgreSQL through co‑design. By evolving both Azure’s infrastructure and PostgreSQL itself, HorizonDB enables the two to work together more efficiently. This talk introduces the architecture of HorizonDB and explores several key design and implementation decisions that enable more scalable, reliable PostgreSQL in Azure. Read More

TopK is a search engine built from the ground up for unstructured retrieval. It combines dense/sparse/multi-vector search, lexical search, powerful filtering, and customizable scoring capabilities in a single, cloud-native system that scales to billions of documents with high ingest throughput and O(10ms) p99 query latencies. In this talk, I'll focus on how TopK is designed on a high-level, including our... Read More

Amazon Aurora DSQL is a distributed SQL database, designed to make it easier to build reliable, scalable, and resilient applications in the cloud, from small prototypes to massive global applications. In this talk, we'll dive into the architecture of Aurora DSQL, looking at how we handle reads, writes, and commits, and talking through the design decisions we made along the... Read More

In this talk, we'll first give an overview of the Oxla analytical database and how it fits into what we do at Redpanda. Then we'll dive into one of the more interesting aspects of Oxla internals: combating memory bandwidth performance bottlenecks in GROUP BY and JOIN via a specialized, custom hashmap implementation. Read More

Modern database management systems (DBMSs) have evolved to support increasingly sophisticated data-intensive applications, at the cost of substantial complexity to configure them for two reasons. First, DBMSs expose a vast configuration space with trillions of possibilities that encompass system knobs, physical design (e.g., indexes), and query options, amongst others. Second, these applications are constantly evolving with changes in data access... Read More

Database management systems (DBMSs) are the foundation of modern data-intensive applications. But as more features are developed to support new workloads, they become increasingly complex and difficult to configure. Thus, researchers have invested decades of effort into autonomous DBMS configuration. Recent advances in machine learning (ML) have produced tools that outperform unassisted experts in real-world deployments. However, these tools are... Read More