Hi! I really like your work, and I've learned a lot from your code ;)
I totally agree that we need more real-world datasets to make better benchmarks and avoid 'overfitting'. It'd be nice to have benchmarks for rich text.
I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this?
Thanks!! I was a bit disappointed diamond types isn't in the table of benchmarking results. Mind if I send you a PR adding it?
I also think it'd be nice to have a native rust benchmarking suite for this stuff. All of the CRDT libraries are either in rust or are moving to rust. Some benchmarks with criterion would be nice to see.
> I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this?
Thats been on my todo list for awhile. I love performance optimization but you can't optimize what you can't measure. I'm just not sure where to get editing traces at the moment given not much software works like this. We could reverse engineer figma enough to record editing traces out of it (or use its plugin API?). Or record some traces from tldraw. Ideally we'd have a few traces from different applications - since I wouldn't be surprised if different apps depend on this stuff differently.
Are you intending to add JSON-style editing support to your CRDT?
If anyone reading this is working on a collaborative app, and wants your app to be exceptionally well supported by CRDTs - please get in touch and we can talk about getting some editing traces. We can't properly optimize collaborative editing systems without them!
PRs are welcome! I wasn't aware that Diamond types also provided a WASM version. I've included it in the native benchmark with Loro though. You can see it here: https://www.loro.dev/docs/performance/native
> Are you intending to add JSON-style editing support to your CRDT?
Loro is another CRDT library I'm working on. It's a superset of crdt-richtext and supports JSON-style editing.
> We can't properly optimize collaborative editing systems without them!
I totally agree that we need more real-world datasets to make better benchmarks and avoid 'overfitting'. It'd be nice to have benchmarks for rich text.
I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this?