Distributed consensus algorithms like Paxos and Raft provide an important building block for distributed systems, but there's a lot more that goes into a resilient and scalable distributed database. CockroachDB's key-value layer is built on many independent and overlapping Raft consensus groups. In this talk I'll explain why we built it this way, and some of the expected and unexpected... Read More

Several enterprises have been producing data not only at high volume but also at high velocity. Many daily business operations depend on real-time insights, therefore real-time processing of the data is gaining significance. Hence there is a need for a scalable infrastructure that can continuously process billions of events per day the instant the data is acquired. To achieve real... Read More

Stream processing has become the dominant processing model for monitoring and real-time analytics. Modern Parallel Stream Processing Engines (pSPEs) have made it feasible to increase the performance in both monitoring and analytical queries by parallelizing a query’s execution and distributing the load on multiple workers. A determining factor for the performance of a pSPE is the partitioning algorithm used to... Read More

InfluxDB is an open source time series database written in Go. This talk will introduce how InfluxDB structures time series data and what makes it different from other use cases like OLTP. We'll then go into the internals of the storage engine we wrote from scratch, the Time Structured Merge Tree, heavily inspired by LSM trees. In addition to the... Read More

For the first time in 25 years, a new non-volatile memory (NVM) category is being created that is expected to be 1000 times faster than current durable storage devices. The advent of NVM will fundamentally change the dichotomy between memory and durable storage in database systems (DBMSs). These new NVM devices are almost as fast as DRAM, but all writes... Read More

The emergence of large main memories and massively parallel processors has triggered the development of multi-core main-memory database management systems (DBMSs). Although the reduction of disk accesses results in low single-thread transaction execution time, scaling these systems on multi-core machines remains notoriously difficult. In particular, the concurrent processing of a large number of transactions can bring about significant performance bottlenecks.... Read More

The structure of real-world complex networks has long been an area of interest, and one common way to describe the structure of a network has been with the k-core decomposition. The core number of a node can be thought of as a measure of its centrality and importance, and is used by applications such as community detection, understanding viral spreads,... Read More

As parallel computing tends toward the exascale, scientific data produced by simulations are growing increasingly massive, sometimes resulting in terabytes of data.  By viewing this data as a dense tensor, we can compute a Tucker decomposition to find inherent low-dimensional multilinear structure, achieving impressive compression ratios with negligible loss in accuracy.  We present recent improvements in our distributed-memory parallel implementation... Read More

One way of understanding the design principles of complex networks is to look at how they are organized at the subgraph level. In this talk I will describe how subgraphs can be seen as fundamental structural units and how they can provide a powerful and very flexible framework for characterizing and comparing networks. I will focus on two concepts geared around... Read More

Most of the academic papers on concurrency control published in the last five years have assumed the following two design decisions: (1) applications execute transactions with serializable isolation and (2) applications execute most (if not all) of their transactions using stored procedures. I know this because I am guilty of writing these papers too. But results from a recent survey... Read More