[¡Databases! 2022] Snowflake Iceberg Tables, Streaming Ingest, and Unistore! (Ashish Motivala)

Why settle for 1 cool db talk when you can get 3? Snowflake is pushing the boundaries of what a unified cloud data platform can do. Today we’ll talk about how Snowflake can be combined with open standards like Apache Iceberg, hard tech to stream data into Snowflake and bring transactional and analytical workloads together in a single platform.

Apache Iceberg is an open source project that provides a way to represent a table as files on the cloud. Iceberg Tables, are a new type of table in Snowflake that allows persisting data and metadata in the Iceberg format. These tables can be processed by any supported query engine like Spark, Trino, Athena, Flink, etc.

Snowpipe Streaming enables users to stream data in a high throughput low latency fashion while preserving ordering and exactly-once guarantees into one or more Snowflake tables and have it be queryable on the order of seconds.

Unistore brings transactional and analytical workloads together in a single platform. Hybrid tables, are a new Snowflake table type enable Unistore workload with support for indexes, constraint enforcement, seamless transaction integration and analytics querying on a single dataset.

This talk is part of the ¡Databases! – A Database Seminar Series.

Bio:

Nileema Shingte is the Tech Lead for Data Lakes at Snowflake. She works on integrating external data seamlessly with Snowflake. In the past she was one of the early major contributors to Presto/Trino.

Tyler Jones is the Tech Lead of Streaming Data Ingestion at Snowflake. Passionate about making data available to data pipelines quickly and incremental processing.

Ashish Motivala is a Founding engineer and Director of Engineering at Snowflake. He's currently working on the Unistore product at Snowflake.

More Info: https://db.cs.cmu.edu/seminar2022#db2