Loading…
Attending this event?
Welcome to Percona Live Online 2021
Online Open Source Database Conference
REGISTER HERE!
Back To Schedule
Wednesday, May 12 • 15:00 - 15:30
Why We Chose Trino. Choosing, Using, and Extending Trino (fka PrestoSQL) For a Primary Datastore

Log in to save this to your schedule, view media, leave feedback and see who's attending!

Video will become available 10 minutes before session start


There are a lot of capture-first pipelines out there, which are very good at squirreling data away, but are relatively slow or cumbersome for queries. For example, Kafka and Pulsar are great for write performance, but horribly slow at scanning all the data in the queue.

For a query-first architecture, a different mindset and approach is required. You can’t build the whole data pipeline and then hope to tune the queries after the fact. Instead, you model the query behaviors you hope to achieve first, and then work backward to define ingestion and indexing requirements. Build it fast, keep it fast. Enter Trino, a distributed query engine that is an ideal starting point for a query-first data architecture like ours. To use it, we built a custom memory connector to use Trino as a primary memory store. An unusual, but fun, use for Trino.

We’ve developed Trino connectors that are optimized to work with local data, so that there is no network hop between the query engine and the data being computed. This gives a 5-20X improvement for our workloads compared with running against even the fastest remote datastores. I’ll walk through the discovery process to get to Trino, and how we built a new Trino connector.

Speakers
avatar for Rob Dickinson

Rob Dickinson

CTO, Resurface Labs
Co-founder and CTO at Resurface Labs, Rob lives and breathes databases. Years at Intel, Dell, and Quest Software, honed his database design skills and his immersion into open source databases was forged by the need to architect and build a scalable solution to solve for customer escalation... Read More →


Wednesday May 12, 2021 15:00 - 15:30 EDT
Room #6
Feedback form isn't open yet.