What kind of delivery guarantees does this offer? And does it provide data replay?
Currently evaluating https://sequinstream.com/ which claims sub-200ms latency, but has a lot of extras that I don’t need and a lighter weight alternative would be nice.
I looked at sequin just now and wish they published their benchmark code. I’m curious how they configure Debezium. At 500 change/s, I get ~90ms average latency with my Debezium cluster after fiddling with Debezium options a bunch.
I don’t love Debezium but I also don’t love Erlang, plenty of us have scars from RabbitMQ’s weird sharp edges…
Sequin engineer here. We'll publish our benchmark repo soon! Indeed, we're still doing a lot of fiddling with Debezium ourselves to make sure we cover different configurations, deployments, etc.
The main thing we want to communicate is that we're able to keep up with workloads Debezium can handle.
(And, re: RabbitMQ, I wouldn't write off a platform based on a single application built on that platform :) )
You can get single digit latency with almost anything for CDC to queue if you properly colocate your services & your CDC ingestion is keeping up with postgres slot
How does it compare against something like Sequin? Looking into something that will allow a cdc event driven workflow but also arbitrarily replaying the event stream on demand.
Sequin sounds like a nice fit for that. I like it and yeah it does have replay.
With pg_flo, I am looking for the tool to be fairly hands off, while declarative at the same time for data filtering, transformation and re-routing like tasks, starting with PostgreSQL as the destination for now. The less time you can spend on cleaning, moving / syncing your data the better. Also quite curious to see what other use cases folks find it useful for.
Replay/reusability has been on my mind and have a sort of an old PR open. Will give it a shot soon.
Honestly Metabase has given best balance between allowing non-technical users to self-service and technical users to dig in and use raw sql if that's what they want. Also it OSS core so you can self host. It is super feature rich and has most everything in the OSS version as long as you don't need enterprise features like SAML auth, audit log, ...,etc
Metabase is the only tool I’ve used where I’ve managed to get non-technical users to actually engage and use to query building tools to answer their own questions.
You would be able to easily offer agents that have all of a tenants data and agent state in a single file which can be synced onto s3.
https://github.com/tursodatabase/agentfs