Stephan Ellner + Lyric Doshi (Google)
Got petabytes to query? Give us a few.
Fetch trillions of rows? Done, what’s next?
Entire data center down? Still fast and strongly consistent.
The Mesa system must serve detailed ads data for reporting, internal audits, analysis, billing, forecasting and more. Meanwhile, Advertisers use the same data to gain fine-grained insights into their campaigns’ performance. Because users include complex enterprise web APIs and interactive web applications, we must answer queries *fast*. Since the data is related to billing and analysis, we must be consistent. And if we lose rows, Google literally loses money. We will also discuss the overall Ads Infrastructure system, general lessons we’ve learned and challenges we face.
Stephan Ellner has been a software engineer in Google’s ads infrastructure team since 2004, focusing on stream processing and low-latency distributed query execution. Stephan currently manages the teams developing the F1 database and related extraction and reporting systems. He holds Master’s degrees in computer science and classical music from Rice University.
Lyric Doshi joined the Mesa team in Google's ads infrastructure team a year ago, where he's currently working on an effort to fundamentally change the data storage model. Before Google, Lyric spent 3.5 years at Vertica exporting features including rewriting the database catalog and locking model as well as leading the project to load and query semi-structured data. He completed his undergrad and Master of Engineering degrees at MIT.
More Info: http://www.pdl.cmu.edu/SDI/2015/012915.html