Revising the approach to dissemination slightly, we will try out having a monthly (or bi-monthly if there is not enough relevant changes for a monthly one) update to the project and sub-projects.
For this round, there’s a new tagged Arcan (i.e. the Display Server) version (0.5.1) and a new tagged Durden (i.e. the example “Desktop Environment”) version (0.2). Although some new features can’t be recorded with the setup I have here, the following demo video covers some of the major changes:
I did not have the opportunity to record voice overs this time around, but here are the rough notes on what’s happening.
The autolayouter is an example of a complex drop:in able tool script that adds additional optional features to Durden. It can be activated per workspace and takes control over the tiling layout mode, with the idea of removing the need for manual resizing/reassignment etc. It divides the screen into three distinct columns, with a configurable ratio between the focus area in the middle and the two side columns. New windows are spawned defocused in a column, spaced and sized evenly, and you either click the window or use the new target/window/swap menu path to swap with the focus area.
It can operate in two different modes, non-scaled and scaled. The non-scaled version acts like any normal tiling resize. The scaled version ‘lies’ to all the clients, saying that they have the properties of the focus area. This means the side- columns get ‘live previews’ that can be swapped instantly without any resize negotiation taking place, reducing the amount of costly resize operations.
You also see a ‘quake style’ drop down terminal being used. This is another drop-in tool script best bound to a keybinding. Its primary use is when you need a persistent terminal with a more restricted input path (keybindings etc. are actually disabled and there’s no activated scripting path to inject input) that works outside the normal desktop environment. In some ways safer than having a sudo terminal around somewhere…
2: Model Window
This is another example drop-in tool script that was ported from the old AWB demo video (the amiga desktop meets BeOS demo from ~2013). What it does is that it simply loads a 3d model, binds to a window and allows you to map the contents of another window to a display part of the 3d model.
There’s clearly not much work put into the actual rendering here, and the model format itself is dated and not particularly well thought out, but serves to illustrate a few codepaths that are the prerequisite for more serious 3D and VR related user interfaces – offscreen render-to-texture of a user-controlled view- and perspective- transform with content from a third party process, with working I/O routing from the model space back to the third party process.
3: Region-OCR to Clipboard
This is an addition to the encode frameserver (assuming the tesseract libraries are present) and re-uses the same code paths as display region monitor, record and share. What happens is that the selected region gets snapshotted and sent as a new input segment to the encode frameserver, that runs it through the OCR engine and puts any results back as a clipboard- style message.
4: Display Server Crash Recovery
We can already recover from errors in the scripts by having fallback applications that adopt external connections and continue from where they left off. A crash in the arcan process itself, would still mean sessions were lost.
The new addition is that if the connection is terminated due to a parent process crash, external connections keep their state and try to migrate to a new connection point. This can be the same one they used, or a different one. Thus, this feature is an important part in allowing connections to switch display servers in order to migrate between local and networked operation, or as a means of load balancing.
5: Path- Activated Cheatsheets
The menu path activated widgets attached to the global and target menu screens were already in place in the last version, but as a primer to the new feature, we’ll show them again quickly. The idea is to have pluggable, but optional, dynamic information or configuration tools integrated in the normal workflow.
What is new this time is the support for target window identity activation. Any external process has an fixed archetype, a static identifier, a dynamic identifier and a user definable tag. The dynamic identifier was previously just used to update titlebar text, but can now be used as an activation path for a widget.
To exemplify this, a cheatsheet widget was created that shows a different cheatsheet based on target identity. The actual sheets are simply text files with a regex- on the first line and empty lines between groups. The widget is set to activate on the root- level of the target menu.
The normal OSC- command for updating window title is used to update the target identity that is used as a selector for the sheet. Vim can be set to update with the filename of the current file and the shell can be set up to change the identity to the last executed command, as shown in the video when triggering the lldb cheat sheet.
6. Connection- and Spawn- Rate Limiting
This another safety feature to let you recover from the possible Denial-Of-Service a ‘connection bomb’ or ‘subwindow-spawn-bomb’ can do to your desktop session. In short, it’s a way to recover from something bad like:
while true; do connect_terminal & done
which has a tendency to crash, 100% live lock or just stall some desktop environments. Here we add the option to limit the amount of external connections to an upper limit or to only allow a certain number of connections over a specified time slice.
7. Dedicated Fullscreen
This feature is still slightly immature and looks like the normal fullscreen but with a few caveats. One is that we circumvent normal composition, so post processing effects, shaders etc. stop working.
The benefit is that we reduce the amount of bandwidth is required. The more important part is what this feature will be used for in the near future, and that is to prioritize bandwidth, latency and throughput to a specified target.
As part of slowly starting to allow 3rd party producers/consumers, there is now an Arcan QEmu display driver (maintained in a separate GIT) that’s at the point where single display video and keyboard / mouse input is working.
The hacky ‘SDL1.2’ preload library has been updated somewhat to work better on systems with no X server available (and there’s an xlib- preload library to work around some parasitic dependencies many has to glX related functions, but it’s more a cute thing than a serious feature).
There is also a SDL2 driver (maintained in a separate GIT) that support Audio/Video/Input right now, but with quite a lot of stability work and quirk-features (clipboard, file DnD, multi-window management) still missing.
Arcan – 0.5.1 Tagged
In addition to the normal round of bug fixes, this version introduces the following major changes:
- Encode frameserver: OCR support added (if built with tesseract support)
- Free/DragonflyBSD input layer [Experimental] : If the stars align, your hardware combination works and you have a very recent version of Free- or Dragonfly- BSD (10.3+, 4.4+), it should now be possible to run durden etc. using the egl-dri backend from the console. Some notes on setup and use can be found in the wiki as there are a few caveats to sort out.
- Terminal: added support for some mouse protocols, OSC title command, bracket paste and individual palette overrides.
- Shmif [Experimental] : Migration support – A shmif- connection can now migrate to a different connection point or server based on an external request or a monitored event (connection dropped due to server crash). This complements the previous safety feature with appl- adoption on Lua-VM script error. The effect is that external connections can transparently reconnect or migrate to another server, either upon request or with external connection adoption on a dropped connection in the event of a server crash. When this is combined with an upcoming networking proxy, it will also be used for re-attachable network transparency.
- Evdev input: (multi) touch- fixes
- Shmif- ext : Shmif now builds two libraries (if your build configuration enables ARCAN_LWA with the egl-dri VIDEO_PLATFORM), where the second library contains the helper code that was previously part of the main platform used for setup for accelerated buffer passing. This will swallow some of the text-based UI code from the terminal. The patched SDL2 build mentioned above requires this lib, and arcan_lwa and game frameserver (with 3D enabled) have been refactored to use it.
Lua API Changes:
- target_displayhint : added PRIMARY flag to specify synch-group membership
- rendertarget_forceupdate : can now change the update- rate after creation
- new function: rendertarget_vids – use to enumerate primary attached vids
- set_context_attachment : can now be used to query default attachment
- system_collapse : added optional argument to disable frameserver-vid adoption
- new function: target_devicehint – (experimental) can be used to force connection migration, send render-node descriptor or inform of lost GPU access
- new function: video_displaygamma – get or set the gamma ramps for a display
- target_seek : added argument to specify seek domain
- New Tools: 3D Model Viewer, Autolayouter, Drop-Down Terminal
- Dedicated fullscreen mode where a consumer is directly mapped to the output device without going through compositing. More engine work is needed for this to be minimal overhead/minimal latency though (part of 0.5.2 work).
- Double-Tap meta1- or meta2- to toggle “raw” window input-lock / release.
- Added display-region selection to clipboard OCR.
- [Accessibility] Added support for sticky meta keys.
- Consolidates most device profiles into the devmaps folder and its subfolders.
- Added a ‘slotted grab’ that always forwards game-device input management to a separate window, meaning that you can have other windows focused and still play games.
- Multiple- resize performance issues squashed.
- Locked- input routing for mouse devices should work better now.
- Basic trackpad/touch display/tablet input classifiers, see devmaps/touch.
- Format- string control titlebar contents
- External connection and window- rate limiting
- Statusbar is now movable top/bottom and the default is top so that those trying things out using the SDL backend won’t be frightened when they are met with a black screen.
- Target- identity trigered cheat sheets
- Button release can now be bound to menu path
The Senseye subproject is mainly undergoing refactoring (in a separate branch), changing all the UI code to use a subset of the Durden codebase, but with a somewhat more rigid window management model.
This UI refactoring along with Keystone based assembly code generation and live- injection will comprise the next release, although that is not a strong priority at the moment.
In addition to further refining the 3rd party compatibility targets, the following (bigger) changes are expected for the next (1-2) releases:
- LED driver backend rework (led controllers, backlight, normal status LEDs and more advanced keyboards)
- Text-to-Speech support
- LWA bind subsegment to rendertarget
- GPU(1) <-> GPU(2) migration, Multi-GPU support
- Vulkan Graphics Backend
- On-Drag bindable Mouse cursor regions
- More UI tools: On-Screen Keyboard, Dock, Desktop Icons