Traces have a very specific data model, and corresponding limitations, which don't really accommodate log events/messages of arbitrary size. The access model for traces is also fundamentally different vs. that of logs.
There are practical limitations mostly with backend analysis tools. OTel does not define a limit on how large a span is. It’s quite common in LLM Observability to capture full prompts and LLM responses as attributes on spans, for example.
> There are practical limitations mostly with backend analysis tools
Not just end-of-line analysis tools, but also initiating SDKs, and system agents, and intermediate middle-boxes -- really anything that needs to parse OTel.
I know the spec says the default AttributeValueLengthLimit = infinity, but...
> It’s quite common in LLM Observability to capture full prompts and LLM responses as attributes on spans, for example.
...I'd love to learn about any OTel-compatible pipeline/system that supports attribute values of arbitrary size! because I've personally not seen anything that lets you get bigger than O(1MB).
Well yeah, there are practical limits imposed by the fact that these have to run on real systems. But in practice, you find that you're limited by your backend observability system because it was designed for a world of many events with narrow data, not fewer events with wider data (so-called "wide events").
OTel and the standard toolkit you get with it doesn't prevent you from doing wide events.
"Wide events" describe a structure/schema for incoming data on the "write path" to a system. That's fine. But that data always needs to be transformed, specialized, for use-case specific "read paths" offered by that same system, in order to be efficient. You can "do wide events" on ingest but you always need to transform them to specific (narrow? idk) events/metrics/summarizations/etc. for the read paths, that's the whole challenge of the space.
You…don’t? This is why tools like ClickHouse and Honeycomb are starting to grow, you just aggregate what you need at query time, and the cost to query is not usually too expensive. The tradeoff is each event has a higher per-unit cost, but this is often the more favorable tradeoff.
> you just aggregate what you need at query time, and the cost to query is not usually too expensive
The entire challenge of observability systems is rooted in the fact that the volume of input data (wide events) on the write path, is astronomically larger than what can ever be directly evaluated by any user-facing system on the read path. Data transformation and specialization and etc. is the whole ball-game. If you can build something directly on top of raw wide-events, and it works for you, that's cool, but it means that you're operating at trivial and non-representative scale.
Logging in OTel is logging with your logging framework of choice. The SDK just requires you initialize the wrapper and it’ll then wrap your existing logging calls and correlate term with a trace/span in active context, if it exists. There is no separate logging API to learn. Logs are exported in a separate pipeline from traces and metrics.
Implementation for many languages are starting to mature, too.
How is logging in OTel?