Computer architectures are moving towards an era dominated by many-core machines with hundreds of cores on a single chip. The decades of exponential single-threaded performance improvement are long gone; hard power constraints and complexity issues have forced chip designers to move from single to multicore designs; clock frequencies have increased exponentially for decades, but now the growth has stopped; and aggressive, out-of-order, super-scalar processors are now being replaced with simpler, single-issue cores. Given the current power limits and the inefficiency of single-threaded processing, increasing the number of cores is currently the only way that architects are able to increase computational power unless a disruptive technology comes along.

Despite this inevitable trend, current database systems are not ready to handle many-core systems. If DBMS designers do not adapt to this reality, then all this computational power will be wasted on database bottlenecks and contention, and the extra cores provided by the hardware will be rendered useless. In particular, it becomes exceedingly difficult for the DBMS to perform concurrency control, logging, and indexing efficiently. With hundreds of threads running in parallel, the complexity of coordinating competing reads and writes to data diminishes the benefits of increased core counts.

The Carnegie Mellon Database Group is collaborating with the MIT Computation Structures Group to develop a novel software-hardware co-design approach for DBMSs in the many-core era. On the software side, rather than attempting to remove scalability bottlenecks of existing DBMS architectures through incremental improvements, we seek a bottom-up approach where the architecture is designed to target many-core systems from inception. On the hardware side, instead of simply adding more cores to a single chip, we will design new hardware components that can unburden the software system from computationally critical tasks.

In this work, we are experimenting with massively parallel on-chip architectures. This allows us to detect and isolate scalability bottlenecks and propose software-hardware co-designed solutions to address those limitations. We accomplish this by running a custom DBMS in the Graphite CPU simulator, a high-performance, parallel simulator for many-core architectures. Running OLTP workloads on these 1000-core simulations with Graphite provides new insights that will enable the development of the DBMS architectures of the future.

People

• Andy Pavlo
• Srini Devadas (MIT)
• Mike Stonebraker (MIT)
• Yu Xia (MIT)
• Xiangyao Yu (Wisconsin)
• George Bezerra (MIT — 2013-2014)

Acknowledgements

This research was funded (in part) by the Intel Science and Technology Center for Big Data and the National Science Foundation (CCF-1438955, CCF-1438967, SPX-1822933).

Publications

1. Y. Xia, X. Yu, A. Pavlo, and S. Devadas, "Taurus: Lightweight Parallel Logging for In-Memory Database Management Systems," Proc. VLDB Endow., vol. 14, iss. 2, pp. 189-201, 2020. PDF Bibtex

   @article{xia2020,
      author = {Yu Xia and Xiangyao Yu and Andrew Pavlo and Srinivas Devadas},
      title = {Taurus: Lightweight Parallel Logging for In-Memory Database Management Systems},
      journal = {Proc. {VLDB} Endow.},
      volume = {14},
      number = {2},
      pages = {189--201},
      year = {2020},
      url = {https://db.cs.cmu.edu/papers/2020/p189-xia.pdf},
    }

2. X. Yu, A. Pavlo, D. Sanchez, and S. Devadas, "TicToc: Time Traveling Optimistic Concurrency Control," in Proceedings of the 2016 International Conference on Management of Data, 2016, pp. 1629-1642. PDF Bibtex

   @inproceedings{yu16,
      author = {Yu, Xiangyao and Pavlo, Andrew and Sanchez, Daniel and Devadas, Srinivas},
      title = {TicToc: Time Traveling Optimistic Concurrency Control},
      booktitle = {Proceedings of the 2016 International Conference on Management of Data},
      series = {SIGMOD '16},
      year = {2016},
      pages = {1629--1642},
      numpages = {14},
      doi = {10.1145/2882903.2882935},
      url = {https://db.cs.cmu.edu/papers/2016/yu-sigmod2016.pdf},
    }

3. A. Pavlo, "Emerging Hardware Trends in Large-Scale Transaction Processing," IEEE Internet Computing, vol. 19, iss. 3, pp. 68-71, 2015. PDF Bibtex

   @article{pavlo15,
      author = {Andrew Pavlo},
      title = {Emerging Hardware Trends in Large-Scale Transaction Processing},
      journal = {IEEE Internet Computing},
      volume = {19},
      number = {3},
      month = {May/June},
      year = {2015},
      pages = {68--71},
      url = {http://www.cs.cmu.edu/~pavlo/static/papers/IC_20150501_May_2015.pdf},
    }

4. X. Yu, G. Bezerra, A. Pavlo, S. Devadas, and M. Stonebraker, "Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores," Proc. VLDB Endow., vol. 8, iss. 3, pp. 209-220, 2014. PDF Bibtex

   @article{yu2014,
      author = {Yu, Xiangyao and Bezerra, George and Pavlo, Andrew and Devadas, Srinivas and Stonebraker, Michael},
      title = {Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores},
      journal = {Proc. VLDB Endow.},
      volume = {8},
      number = {3},
      month = {November},
      year = {2014},
      pages = {209--220},
      publisher = {VLDB Endowment},
      url = {https://www.vldb.org/pvldb/vol8/p209-yu.pdf},
    }

5. X. Yu, "An Evaluation of Concurrency Control with One Thousand Cores," Master's Dissertation , 2015. Bibtex

   @mastersthesis{yu2014master, title={An Evaluation of Concurrency Control with One Thousand Cores},
      author={Yu, Xiangyao},
      school={Massachusetts Institute of Technology},
      year={2015},
    }