The next major release of Tributary will support Avro, Protobuf and JSON along with the Schema Registry it will also bring the ability to write to Kafka with transactions.
But really you should get excited for DuckDB Labs to build out materialized views. Materialized views where you can ingest more streaming data to update aggregates. This way you could just keep pushing rows through aggregates from Kafka.
It is going to be a POWER HOUSE for streaming analytics.
Exactly. I have also been playing with DuckDB for streaming use cases, but it feels hacky to issue micro-batching queries on streaming data in short intervals.
DuckDB has everything that streaming engines such as Flink have; it just needs to support managing intermediate aggregate states and scheduling the materialized views itself.
Is this to be used in an analytics application backend sort of scenario?
I am familiar with materialized views / dynamic tables from enterprise-grade cloud lake type offerings, but I've never quite understood where duckdb, though impressive, fits into everyones use case. I've toyed with it for personal things, it's very cool having a local instance of something akin to snowflake when it comes to processing and aggregating on Big Data™ but generally I don't see it used in operational settings. For application development people are generally tied to sqlite and postgres.
It all does seem really cool though, I guess I'm just not feeling creative enough to conjure up a stream-to-duckdb use case. Feel free to bombard me with cool ideas.
I would guess there aren't as many because, in order to implement this, your language must already have an HTTP/3 library. My language of choice doesn't even support QUIC yet (so I'm writing the library for it, then for HTTP/3). I wouldn't be surprised if other languages are similar.
As of one month ago, Java still didn't have HTTP/3 support. Though it's apparently coming in March (with JDK 26).
But yeah the HTTP/3 integration definitely makes WebTransport harder to support. The QUIC connection needs to be shared between HTTP/3 and WebTransport.
Yet again Hollywood is prescient. This post reminds me of the language of the aliens in Arrival. It seems like the OP would see that as a reasonable input to an LLM.
Without diversification, we’re just setting ourselves up for an economic rug pull the moment the AI growth hits the law of large numbers and people realize its not the magic they think it is. Economically? Don’t worry it’ll just be another “oops, your 401k lost a bit, here’s a bailout” moment. AKA, stimulus with sprinkles.
I'm not a biologist, but I grew up in West Yellowstone around the time wolves were reintroduced. Their return—and its impact—has been extensively studied by experts far more qualified than me.
That said, I believe wolves had a profound effect on the Yellowstone ecosystem, particularly on elk and deer populations. Before their reintroduction, those species had few natural predators beyond hunters, vehicles, bears, and the occasional mountain lion. The imbalance led to overgrazing and the spread of diseases like Chronic Wasting Disease (CWD) in elk.
