DuckCon #7
State of the Duck
Hannes Mühleisen, Mark Raasveldt · DuckCon #7
7
sections
16
slides
~28 min
duration
01 / Welcome & Mission
DuckCon #7 opens: a global, sponsor-backed community
- Seventh DuckCon, live-streamed worldwide so people who cannot travel to Amsterdam can still take part.
- Audience Q&A runs through the Slido system at qa.duckcon.org; questions are upvoted and prioritized live.
- Thanks to sponsors: MotherDuck (gold) and Spiral (silver), who make the free event possible.
- The company mission stays the same: turn 'data fear' into 'data confidence' when working with data of any shape or size.
More detail
Hannes frames the mission around the everyday anxiety of messy data: folders of thousands of wonky JSON files, stray CSVs, intractable Parquet. DuckDB exists to make working with all of it approachable.
02 / DuckDB Today
From an in-process SQL library to a full data ecosystem
- DuckDB is 'built for your stack': native clients and integrations across protocols, storage systems, other databases, platforms, and data formats.
- It has effectively become a universal database client, running on an ever-growing list of platforms and languages, with heavy external contribution.
- Adoption is booming: over 1 million installs per day and more than 160 million extension installs per month (traffic donated by Cloudflare).
- Strong social and ranking momentum too (GitHub stars, LinkedIn followers, DB-Engines), and DuckDB is 'loved by agents' as well as humans.
More detail
Making all those formats work required people to become deep, sometimes reluctant, experts in CSV, the full Parquet spec, Avro, and more.
03 / DuckLake
The lakehouse, radically simplified
- DuckLake takes the lakehouse idea and strips away layers of indirection: a catalog plus data files, instead of many nested metadata/manifest layers as in Iceberg.
- It went from a '0.1' wild idea just over a year ago to a production-ready DuckLake 1.0 released a few weeks before the talk.
- Extension downloads for DuckLake are now on par with Iceberg and Delta within DuckDB, within roughly one year.
- DuckLake has grown into a second, grown-up project alongside DuckDB, with multiple implementations beyond DuckDB itself.
More detail
The signature comparison slide shows Iceberg's deep catalog/metadata/manifest/data-file tree on one side versus DuckLake's flat catalog-to-data-files shape on the other.
04 / Quack & DuckLabs
DuckDB talks to DuckDB, and the company renames itself
- Quack is a new communication protocol/extension that lets two DuckDB instances talk to each other: one runs 'quack serve', the other attaches.
- It brings a client-server pattern to DuckDB; people already built standalone Quack clients within weeks of its preview.
- Quack 2.0 will add cleaner syntax via a CONNECT statement to forward queries directly, replacing the verbose remote.query(...) form.
- With DuckDB, DuckLake, and Quack all in play, the company rebranded from DuckDB Labs to DuckLabs.
More detail
The code slide contrasts the server side (quack_serve, CREATE TABLE) with the client side (CREATE SECRET, ATTACH 'quack:localhost', then FROM remote.fuu).
05 / Year of DuckDB as a Server
DuckDB 2.0 (Cinnamon Teal): the server era
- After the 'year of the lakehouse' (DuckLake, Iceberg, Delta), the next theme is the 'year of DuckDB as a server', with DuckDB 2.0 shipping in the fall.
- Running DuckDB long-term raises new needs: better metrics, logs, observability, stability, and true multi-tenant operation.
- DuckDB has had ACID transactions, multiple connections, MVCC, and isolation since day one; the client-server shift finally makes those features central.
- Benchmarks (first shown when Quack was revealed) suggest DuckDB is competitive with Postgres on many transactional workloads, enabling hybrid analytical + transactional use.
More detail
The isometric diagram shows many DuckDB nodes issuing INSERTs into a central server while a laptop runs SELECTs, illustrating the multi-writer, client-server scenario Quack enables.
06 / DuckDB 2.0 Features
Variant, triggers, async IO, partitioning, and a fresh core
- Variant type (already in v1.5): 'JSON on steroids' that infers structure, compresses/optimizes storage, and speeds queries; a future goal is to back the JSON type with it. Great for real-time log ingestion.
- SQL triggers: fire follow-up events (e.g. write to an audit table after an insert), useful for long-running services and as a building block for internal features.
- Asynchronous IO decouples the IO layer from query processing, adding parallelism for remote reads (Parquet first) to make object-store/network access much faster.
- Partition-aware execution makes query planning/optimization exploit partitioning in DuckLake, Iceberg, and Parquet-on-S3 for faster scans.
- Under the hood: a broader stable C extension API (C++ V2) so extensions are built once and keep working, plus a brand-new in-house parser replacing the Postgres parser.
More detail
The stable API aims to fix community-extension churn, where extensions had to be rebuilt per DuckDB version or risk disappearing. The new parser is meant to be backwards compatible and to let extensions add SQL syntax more easily.
07 / Q&A
Closing questions and the five-year vision
- On big vendors (e.g. Databricks' Raiden) building DuckDB-like in-process engines: the team is flattered; imitation is the sincerest form of flattery.
- Postgres compatibility, including the spatial extension moving closer to PostGIS, is a high priority; they avoid deviating from Postgres without good reason.
- On limited Google Cloud integration: a nudge to raise it with your Google sales rep, as they are happy to collaborate.
- Five-year vision: 'DuckDB everywhere', pursuing (perpetually five-years-away) world domination by expanding where DuckDB can run.
More detail
Hannes and Mark field upvoted Slido questions together to close the session before handing off to the next speaker.
Key Takeaways
- DuckDB has outgrown its 'in-process SQL' origins into a broad ecosystem: 1M+ installs/day and 160M+ extension installs/month.
- DuckLake reached a production-ready 1.0 within a year and already rivals Iceberg and Delta in DuckDB extension downloads.
- Quack introduces DuckDB-to-DuckDB communication and a genuine client-server pattern, with cleaner CONNECT syntax coming in 2.0.
- The company rebranded to DuckLabs to reflect a portfolio beyond DuckDB (DuckLake, Quack).
- DuckDB 2.0 (Cinnamon Teal), due in the fall, centers on the 'DuckDB as a server' theme: observability, stability, and multi-tenancy.
- Day-one ACID/MVCC transaction support makes DuckDB surprisingly competitive with Postgres for hybrid transactional + analytical workloads.
- Key 2.0 features: variant type, SQL triggers, asynchronous IO, partition-aware execution, a stable C extension API, and a new in-house parser.