> User comes to an artistic website whose author thinks purple on red is a great color schema.
The outer app used as your desktop/browser-UI has two mechanisms at its disposal. The first is collaborative - it can convey the preferred colour-scheme so the 'page' you are browsing to can pick accordingly. That's actually how the terminal emulator and non-terminal shell both gets their colour schemes.
The other is enforced - just as tone mapping is needed on the composition level to deal with HDR, you can use the same facility to apply colour grading per 'page'. That's what I do to chrome right now without it knowing. If the average luminosity exceeds a threshold, it picks a colour map that enforces a dark mode regardless of what the browser wants. Similarly, if a game doesn't respond to resize requests but continues to provide a low 256x240 like output I apply an NTSC signal degradation shader pass into a CRT simulation one. By having it as a composition effect means that I can have my own view of colours, but if I 'share desktop' to someone else the original form is retained.
> - User opens the very newest website, which uses all the latest website standards, including new video codec, new font and new text rendering effect. (In regular web, this would only happen once most clients update to the very latest version of the browser, and telemetry says enough clients have updated)
It's similar to Android API_LEVELS. API breaks (we've had, I think two, since 2013) have been handled with a hook script that exposes the old behaviour. New video codec deployment is a bit special and there's a protocol level degradation path for that with a few nuances. It has mainly been used for hardware accelerated web cameras as a remote data source where it starts with direct pass-through of whatever streams it provides. If the client fails to decode (stream-cancel:reason=codec failure) it signals that and the source end reverts to a safer default from a hierarchy (h264 into zstd).
> In current web, I'd use "reader mode", or a custom UserScript if I plan to go to website often. What happens in Arcan - is this up to the "decode client"? What if the site author chose a "decode client" which has no such functionality?
Decode can be used for that but it is overkill. The hook-script facility (arcan -Hmyscript1.lua -Hmyscript2.lua myapp) can interpose every function the app has access to transparently. Reroute patterns like "load_image_asynch("something.jpg") into null_surface() there. These and other options like the set of encode/decode/permitted integration with allow-list of local software can* be preset per authentication key.
There is a PoC 'more than adblock' that uses an LLMed version of 'decode' in my bin of experiments. The first part is a hookscript that routes all image loading through a version of decode with a local classifier model asked to 'describe the contents of this image and tell me if it looks like an advertisment'. If not the default path applies. If yes it substitutes an image of puppies playing tug of war with a rubber chicken.
Right, I've realized that I asked too many questions in this thread, so I tried to ask more focused question here. You did not answer it, so let me as the most interesting (to me) question:
>> If the answer to the question above is "Arcan will use a different external client for each", then how will those clients be managed? Is there going to be a allow-list of video codecs and text renderers hardcoded into the browser? Or any site can specify any requirement, and browser will just go fetch it automatically on first visit? ...
Since you mention "API breaks", it sounds like list of clients is hardcoded in browser version. So what is the innovation compared to existing web, _from the standpoint of the website author_? I see the the following items:
- Discrete "API Levels" instead of forever backwards compatibility. More bundled libraries.
- Lua instead of Javascript, no WASM or any other low-level/compiled code
- No "HTML equivalent" - if a page wants to do text layout, then it has to pull layout library
- Bundle all libraries in initial package instead of loading them dynamically
- No third-party storage, and maybe first-party is optional too (not sure what's the plat for this.. user prompt? explicit action?)
Did I get this right? did I miss something?
(From what I see, the SHMIF, pledge, Decode/Encode/Network/Terminal, and all this stuff is basically internal browser implementation detail - cool to know if you are a browser author, but does not matter for your typical page writer, right?)
The entire article is from the browser internals angle. The page-writer one comes when we have tools to compile pages into app and layer in user features. The rough path for that is simpler ones first (Sile typesetter and Pandoc), then more exotic ones. An example of the later would be a "print to" modification to a browser that can preserve more content like animations and audio and not be restricted to fixed sized pages.
It is forever backwards compatible. API levels is for the rare few cases where we need to deprecate or break something on the engine side but can apply a runtime fix-up script to replicate the behaviour on the scripting level.
Decode/Encode/Network/Terminal is user-relevant insofar as the set of those are interchangeable binaries. If you want more or less (paranoia, legal environment restrictions) media format support, export or streaming capabilities, etc. those can be swapped out per site without any changes to the engine. What is being considered is to be explicit about expected formats in the manifest so that incompatibilities can be detected.
SHMIF is user relevant insofar as "wasm or low-level compiled code" running as local software can be embedded and controlled by the app, allow-listed by the user.
> No third-party storage, and maybe first-party is optional too (not sure what's the plat for this.. user prompt? explicit action?)
> User comes to an artistic website whose author thinks purple on red is a great color schema.
The outer app used as your desktop/browser-UI has two mechanisms at its disposal. The first is collaborative - it can convey the preferred colour-scheme so the 'page' you are browsing to can pick accordingly. That's actually how the terminal emulator and non-terminal shell both gets their colour schemes.
The other is enforced - just as tone mapping is needed on the composition level to deal with HDR, you can use the same facility to apply colour grading per 'page'. That's what I do to chrome right now without it knowing. If the average luminosity exceeds a threshold, it picks a colour map that enforces a dark mode regardless of what the browser wants. Similarly, if a game doesn't respond to resize requests but continues to provide a low 256x240 like output I apply an NTSC signal degradation shader pass into a CRT simulation one. By having it as a composition effect means that I can have my own view of colours, but if I 'share desktop' to someone else the original form is retained.
> - User opens the very newest website, which uses all the latest website standards, including new video codec, new font and new text rendering effect. (In regular web, this would only happen once most clients update to the very latest version of the browser, and telemetry says enough clients have updated)
It's similar to Android API_LEVELS. API breaks (we've had, I think two, since 2013) have been handled with a hook script that exposes the old behaviour. New video codec deployment is a bit special and there's a protocol level degradation path for that with a few nuances. It has mainly been used for hardware accelerated web cameras as a remote data source where it starts with direct pass-through of whatever streams it provides. If the client fails to decode (stream-cancel:reason=codec failure) it signals that and the source end reverts to a safer default from a hierarchy (h264 into zstd).
> In current web, I'd use "reader mode", or a custom UserScript if I plan to go to website often. What happens in Arcan - is this up to the "decode client"? What if the site author chose a "decode client" which has no such functionality?
Decode can be used for that but it is overkill. The hook-script facility (arcan -Hmyscript1.lua -Hmyscript2.lua myapp) can interpose every function the app has access to transparently. Reroute patterns like "load_image_asynch("something.jpg") into null_surface() there. These and other options like the set of encode/decode/permitted integration with allow-list of local software can* be preset per authentication key.
There is a PoC 'more than adblock' that uses an LLMed version of 'decode' in my bin of experiments. The first part is a hookscript that routes all image loading through a version of decode with a local classifier model asked to 'describe the contents of this image and tell me if it looks like an advertisment'. If not the default path applies. If yes it substitutes an image of puppies playing tug of war with a rubber chicken.
* code being written as we speak.