Arcan versus Xorg – Approaching Feature Parity

This is the first article out of three in a series where I will go through what I consider to be the relevant Xorg feature set, and compare it, point by point, to how the corresponding solution or category works in Arcan.

This article will solely focus on the Display Server set of features and how they relate to Xorg features, The second article will cover the features that are currently missing (e.g. network transparency) when they have been accounted for. The third article will cover the features that are already present in Arcan (and there are quite a few of those) but does not exist in Xorg.

It is worthwhile to stress that this project in no way attempts to ‘replace’ Xorg in the sense that you can expect to transfer your individual workflow and mental model of how system graphics works without any kind of friction or effort. That said, it has also been an unspoken goal to make sure that everything that can be done in an Xorg environment should be possible here — in general there is nothing wrong with the feature set in X (though a bit limited), it is the nitty gritty details of how these features work, are implemented and interact that has not really kept up with the times or been modelled in a coherent way. Thus, it is a decent requirement specification to start with – just be careful with the implementation and much more can be had to a fraction of the code size.

The very idea of the project as a whole is to find new models for system graphics, preferably those that can survive the prospect of a coming ‘demise’ of the ‘desktop’ as the android/chrome alphabet soup monster keep slurping it up, all the while established ones make concessions after concessions to try and cater to a prospective user base that just want things to “work” at the expense of advanced user agency.

In terms of the ‘classic’ desktop, Xorg with DRI3 work is already quite close to that dreaded ‘good enough’, and the polish needed is mostly within reach. My skepticism is based on the long standing notion that it won’t actually matter or be anywhere near ‘good’ enough.

This article is about as dry an experience as quaffing a cup of freshly ground cinnamon – even after the wall of text was trimmed down to about half of its original length. To make it less painful to read, you can use these shortcuts to jump between the different feature categories.

Most of these section are written in a way that they start with a summary of the X(org) perspective, and then contrast it with how it is handled in Arcan.

As a precursor to the other sections, lets first take a look at two big conceptual differences. The first one is that in Arcan, the only primitive a client really works with is referred to as a ‘segment’. A segment is a typed IPC container that can stream audio and/or video data in one direction, ‘client to server’ or ‘server to client’. It also carry associated events, metadata and event-bound abstract data handles (e.g. file descriptors) in both directions.

The type is a hint to/from the policy layer (which is somewhat similar to a ‘Window Manager’), e.g. popup, multimedia, clipboard, game and so on – used to assign priority and influence decisions such as suppress-suspend or input grabs without letting clients request such state changes, which is otherwise the annoying default in many other environments.

Any connected client gets one segment (the ‘primary’) and closing that one will release all resources associated with the client. A client may negotiate for additional segments, but these can be rejected, and rejection is, just as in real life, the default behaviour. The policy layer has to explicitly opt in to further allocations. A client may also negotiate for extending the capabilities of the segment to support advanced and privileged features, such as access to VR related devices, hardware colour lookup tables and so on.

A second key difference is the aforementioned policy layer. This layer act as both the window manager and event driven I/O “routing” rules combined into one. It is not treated as a normal client, but instead has access to a unique set of privileged functions that can be leveraged to explicitly control everything from display synchronisation, source to output mapping, to event propagation and synthesis.

The premise is that the choice in policy layer should be modifiable at the user’s behest. It can be thought of as a firewall and router for all UI related activity – in control of what goes where, when, why, and in what shape. This layer also have the option to not only accept and map client segments, but also to explicitly push a segment to a client, as a means of probing- and announcing- support for some features.

Clients and Privileges

For those not entirely comfortable with display server parlance, recall the distinction between X11 (X protocol version 11) and Xorg. X11 is a protocol (a literal stack of papers), while Xorg is the current de facto server side implementation of that protocol – though the protocol itself only expose a subset of what the server itself can potentially do.A reasonable parallel is the one of the relationship between HTTP (or SPDY, QUIC) and the rest of the web browser (Firefox, Chrome, …) both in engineering effort and relative importance. For clients to communicate using the protocol, the de facto implementation comes via the xlib library or via the more recent ‘xcb’ library.

This is a setup that is quite rare for user-facing operating systems in general: Windows does not have a display server protocol, and the same goes for OSX and Android. The real value of a protocol is communication between hard system barriers, politics condensed so to speak, but interaction between userspace components is not a particularly hard boundary. The protocol distinction is also not very common in related subsystems like audio. What happens instead is that they do expose APIs for letting other clients interact and integrate with the outer system, but that, by its very definition, only describes the interface – nothing strictly about the communication channel, ordering rules, valid- and invalid- bit patterns or actions to take on non-compliance, as some of those parameters are regulated on an OS level in the shape of the lesser known ABI.

In a similar vein, Arcan does not have a protocol for normal clients as such, and compatibility support for protocols like X11 and Wayland come via translation services built as separate opt-in special clients for stronger separation, sanity and sandboxing reasons.

API-wise, Arcan exposes three APIs. ‘Shmif’, ‘Tui’ and ‘Alt’. The philosophy behind the design is to let the server side absorb systemic complexity, and strive to make clients as trivial as possible to both develop and inspect. No multi-million lines of code toolkit should ever be necessary. The perspective is to instead emphasise the extremes, that is to focus on the small “one task” kind of clients and on integration with bigger “embedded universes” like virtual machines and web browsers, and leave the middle to fester and rot.

The ‘shmif‘ API provides the low level interface and exposes a large set of features over a deceptively simple setup, one that was mostly inspired by old arcade hardware designs. This API is kept in lock-step with the engine. It may have incompatible changes between versions (but so far, not really), and is kept in lock step with the main engine up until the fabled ‘1.0’ release. This means that the running engine version needs to match that of the shmif API version. It enforces the attitude of “if you upgrade the server side, upgrade the clients” as well. Therefore, it does not provide means for “extensions”. The reasoning for this is that it should be absolutely clear from a version which features that are present or not. It should not be left up to guesswork, a “TERM=xxx ; termcap” style database or Wayland- “Charlie Foxtrot” style extension registry.

The ‘tui’ API builds upon shmif and primarily strikes at text-oriented user interfaces as a means of dissolving the dependency these have had traditionally to terminal protocols and the need to go via a terminal emulator. This opens the door to better integration with the window management style in the outer desktop; sane input models; removes the need for tmux style multiplexers; multimedia- capable command line tools and so on.

Lastly, the ‘alt’ API is the high level engine interface itself and is currently exposed via Lua (see also: AWK for multimedia). It is used server side to implement the policy layer, but it can also be used client side via a special Arcan build (referred to as LWA) that acts as a normal 2D/3D renderer. Re-exposing the same API entry-point over a line-format is one of the missing components scheduled for the ‘at feature parity’ part in this articles series.

Going back to the arcan-shmif API — it is used to implement support for a number of features, but also for supporting other protocols. These typically come as separate processes and services that you switch on and off during runtime. The big two examples, right now, being ‘waybridge’ for Wayland support and ‘xarcan’ for X11 client support. These are kept as separate, privilege separated and, optionally, sandboxed processes and can run in all combinations: one bridge to one client, one bridge to many clients and many bridges to many clients.

Programs that use the arcan shmif API are split into three distinct groups: ‘subjugated‘, ‘authenticated‘ and ‘non-authoritative‘ – collectively referred to as ‘frameservers‘. The authenticated clients are started through the initiative of the window manager and their parameters are predefined in a database. They inherit their respective connection primitives, thus keeping the chain of trust intact.

Subjugated clients have a preset ‘archetype‘. This archetype regulates its intended role in the desktop ecosystem, e.g. media decoding,  encoding, CLI shell etc. and the window manager may forcefully create and destroy these with predictable consequences. They can therefore be considered as a form of ‘do one thing well then die’ graphical clients. The big thing with this is that a window manager can make stronger assumptions about what to make of their presence or lack thereof.

The last ‘Non-authenticated, non-subjugated’ clients are ones that more closely match the model seen both in other Wayland compositors and in X11 (X has a very mild form of authentication, Wayland completely fails in this regard) and is simply some client started from a terminal (tty) or, more likely, pseudoterminal (ptty) using some kind of connection primitive and discovery scheme.

In X11, clients connects and authenticates to an X11 server with a flat (disregarding certain extensions, there are always ‘extensions‘) set of quite far-reaching permissions – clients can iterate, modify, map, read and redirect parts of both the event routing and the scene graph (a structure that controls what to draw, how to draw and when to draw). It is a tried and true way of getting very flexible and dynamic clients – which is also its curse.

In Arcan, there is a minimal default set of permissions which roughly corresponds to drawing into a single re-sizeable buffer, along with a bidirectional event queue. Everything else is default-reject policy, and has to be opted in. This includes attempts to retrieve additional buffers, on the principle that a client should have a hard time to issue commands that induces uncontrolled server-side resource allocations. Partial motivation for the resource stringency is that the display server can run for extended periods of time, and small performance degradation from heap fragmentation and so on- accumulate slowly, presents itself as ‘jankyness’, ‘judder’, ‘lag’ and can be terrifyingly hard to attribute and solve.

The upside is that the ‘window manager’ can extend this on a sliding scale from accepting custom mouse cursors for certain clients, to allowing GPU accelerated transfers, access to colour controls, buffer readbacks and unfiltered event-loop injection. Principally, albeit unwise, there is nothing that stops this layer from opening the permission doors wide open immediately and just replicate the X11 model.

Window Managers (WM)

One of the hallmark features of any decent display system is to be able to allow the user to manipulate, tune, or replace the interface, event routing and event response as he sees fit.

In X, any client can achieve this effect, which is also one of its many problems – if one or many clients tries to manage windows you have the dubious pleasure of choosing between race conditions (which manifest as short intermittent graphics glitches in conflicted territories like decorations, or buffer-content tearing) or performance degradation in terms of increased latency and/or lowered throughput. In many cases, you actually get both.

Arcan keeps the Window Manager concept as such, but it is not treated as ‘just another client’ with access to the a same set of mechanisms as all the others. Instead, it act as a dominant force with access to a much stronger set of commands. Since the engine both knows and trusts the window manager, it can allow it to drive synchronisation and get around the synchronisation problems inherent to the X approach.

All example window management schemes that have been showed in the videos here over the years are implemented at this level and they have acted as driving forces for evolutionary design of the scripting API. Some of the high level scripts, mouse cursor management being one example, gets generalised and added to a shared set of opt-in ‘builtin’ scripts.

Because of these choice, most of the features that the individual schemes provide can be transplanted into other window managers at a much lower developer effort than what would be ever be possible in Xorg or any other current Wayland compositor for that matter.

Displays vs Connection Points

In X11, a client connects to a ‘display’ specified with the DISPLAY=[host]:[.screen] environment variable. This is an addressing scheme that maps to a domain or network socket through which you can connect to the server.

In Arcan, the environment variable is ARCAN_CONNPATH=”name” which also points to a domain socket (unless the connection is pre-authenticated and the primitives are inherited). The big difference is that the “name” is set by the Window Manager and it is consumed on use, meaning that when a client has connected through it, it disappears until it is reopened by the window manager.

This is a very powerful mechanism as it allows the Window Manager to treat clients differently based on what address they connect to. It means that the API can focus on mechanisms and not policy, and the Window Manager can select policy based on connection origins. Instead of just using the name as an address to a server, it is an address to a desktop location.

Trivial examples would be to have singleton connection points for an external status bar, launchers or for the desktop wallpaper – without having to pollute a protocol with window management policies and worse, modifying clients to enable what is principally window manager scheme dependent policy.

This mechanism has also been used to successfully implement load balancing and denial of service protection (something like while true; do connect_client &; done from a terminal should not be able to render the UI unresponsive or cause the system to crash or terminate).

Reading Window or Screen contents

An X client has a few options for reading the contents of a window, its own or others. This can be used to sample whole or parts of the display server output, but at a fairly notable cost. The actual means vary with the set of extensions that are available.

For instance, if GLX (the GL extensions to the X protocol) is available, the better tactic is to create an offscreen “redirected” window on the server side, bind that to a client accessible texture and draw/copy the intended source into it and then let the GPU sort out the transfer path.

In Arcan, the feature is much more nuanced. A client can request a subsegment of an ‘output’ type, but it has no explicit control over what this means more than “output intended for the client”. The same mechanism is used for the drop part in drag and drop, and the paste part in clipboard paste. The window manager explicitly specifies which sources and which transformations should be sampled and forwarded to the client.

This means that some clients may be provided full copies of the screen, or something else entirely, and the contents can be swapped out at will and at any time, transparently to the client. Since each segment can contain both audio and video, the same approach applies to audio. The philosophy is that the window manager provides user controls for defining selection and composition, and the clients provides processing of composited contents.

The window manager can also chose to force-push an output segment into a client as a means of saying “here’s output that the user decided you should have, do something with it”. The client gets the option to map it and starts reading, or just ignore and implicitly say “I don’t know, I don’t care”. A useful mental model of this is like normal ‘drag and drop’ but where the drag contents is user- defined, and the target is unaware of content origins.

Input and Input Injection

The input layer is a notoriously complicated part of X11. The specs for XInput2 and XKB alone are intimidating reads for sure, not including deprecated but present methods and so on. They also relate to one of the larger issues with X11, being the lack of WM mandated input coordinate space translation and scaling. You have access to zero race condition free ways of controlling who gets control over what input subsystem and when, which is a notable cause of all kinds of problems – but it also severely hinders the features a desktop can provide.

Not only is the input model complicated, but it is also limited. A lot of things has happened to input devices in terms of both speciality accessibility, gaming devices and hybrid systems like VR positioning and gesture input.

The philosophy in Arcan is to have the engine platform layer gather as much as it can from all input devices it has been given permission to use, and provide them to the window manager scripts with minimal processing in between – no acceleration or gesture detection. This is not always reasonable for performance reasons, which boils down to the sample rate of the device and the propagation cost for a sample. It is therefore possible to set rate limits or primitive filters as to not oversaturate event queues – but otherwise the input samples are as raw as possible.

This is done in order to defer decisions on what an input device action actually means. For that reason, there are default scripts the window manager can chose to attach which implements gesture detection and similar higher level features. These parts combined fulfil the input manipulation, remapping and synthesis criteria.

For input injection, things are even more flexible. The “event model” in the arcan-shmif API match the one the engine uses to translate between the platform layer (where the input events from devices normally originate) and the scripting layer. Client event processing simply means that the event queues in the memory shared with each client gets multiplexed and translated onto a master queue, with a priority and saturation limiter.

A key difference however is that the events provided by a client gets masked and filtered before they are added to the master queue. The default mask is very restrictive, and the events associated with input devices are blocked. The scripting API, however, provides target flags, allowing this mask to be changed. Combine this with the pre-authenticated setup and other programs can act as input drivers in a safe way, yet still present UI components (such as on-screen keyboards) or configuration interface. This code example illustrates a trivial input injector which presses random keys on a random number of keyboards.

Displays, Density and Colour Management

X has the XRANDR extensions for exposing detailed information about output displays. This interface is also used for low level control of resolution, orientation, density and accelerated colour lookup tables.

For colour lookup tables, there are at least two big drawbacks.  The first is that any client can make a display useless by providing broken or bad tables. The second is that there is no coordination about which client knows what or in which order table transforms are to be applied or when – a game might need its brightness adjusted while an image viewer needs more advanced colour management. This will become more pronounced as we drift towards a larger variety in displays, where HDR will probably prove to be even more painful to get right than variations in density.

Exposing low level control of the other display parts comes with the problem that a lot of extra events and coordination is needed for the window manager to understand what is happening, and these solutions worked at a time when CRT displays were at their zenith and not as well now when their collective sun is setting.

Density is a related and complicated subject. A decent writeup of the interactions between X extensions and displays in the context of density can be found here: [Mixed DPI and the X Window System], and it is a better read than what can be covered here.

As for Arcan, there is an extended privileged setup of the SHMIF API where a client gets accessed to enumerating displays and their properties in an RANDR- like way. It can also be used to both retrieve and submit acceleration tables, but it ultimately requires that the window manager opt in and act as an intermediate in order to resolve conflict. Such tables are therefore tracked and managed on a per client level and not per display,.

For resolution and orientation control, see the section on ‘Management Interfaces‘ section further below, and for more details on Density, see the ‘Font Management‘ section. What is worth to mention here is that these properties are all handled ‘per client’ and the window manager is responsible to forward relevant information on a client basis. Thus, multiple clients can live on the same display but have different ideas on what the density and related parameters are.

Font Management

Font rendering and management is a strong candidate, if not a sure winner, for the murkiest part in all of X – though the input stack also come close (since the print server is principally dead). The better description that can be found is probably keithp’s “The Xft Font Library” but it also relies on quite some knowledge on the huge topic of text rendering and how it relates to network transparency and a historical perspective on printing protocols and printers as “very high on DPI, extremely low on refresh rate” displays. The topic of X Font Management in the shape of XFS (the X Font Server) and XFLD (X logical font description) will be left in the history department on account on it practically being deprecated, but there is another aspect that is worth bringing up.

Text is one of the absolutely most important primitives being passed around. No doubt about that. Fonts are connected in order to transform text into something we can see.

This transform depends on the output display, such as its density, subpixel layout and orientation. It also requires user context sensitive information such as presentation language in order to provide proper shaping and substitutions.

As a short example of a ‘simple’ problem here: take subpixel hinting where the individual colour channels are biased in order to provide the appearance of higher text resolution. While it may look better locally, taking a ‘screenshot’ or ‘sharing’ such a window with a remote viewer has the opposite effect.

The division of responsibility for text processing is an intractable problem with no solution in sight that doesn’t have far reaching consequences and quality tradeoffs. The legacy compromise is to work in multiples of 90, 96 DPI or similar constants and then express a scale factor. For new designs, this should make every high school maths and physics teacher cringe in disgust.

In Arcan, a client is currently responsible for all text drawing while the window manager suggests some of the parameters. The way it is managed is that at the moment a connection is opened, the client immediately gets a set of initial properties that includes a set of fonts (primary, secondary etc.), preferred font size, output display properties and language. These can be dynamically overridden later via a handful of events, primarily FONTHINT (size or hinting changes, transfer of new descriptors), DISPLAYHINT (preferred output size and target density) and GEOHINT (position and input-output languages).

Management Interfaces

This category is a bit more eccentric than the others in that for Xorg, a lot of ‘management’ behaviour can be exposed via clients and the quintessential tools that people associate with this is either part of RANDR or via xdotool which more falls into ‘creative use of XTEST and other game mechanics’. The randr- like management is covered in the section on displays

Some of those roles are split up and covered in the other sections on display management and input injection. Due to the increased role of the window manager, most of the other automation controls have to go through it.

The compromise is that there are support scripts that can be pulled in by the window manager which expose a data model for exposing window management controls ‘as a file system’, along with a support FUSE (demo) driver that allows this file system to be mounted so that the normal command line tools can be used to discover, control and script higher level management in cooperation with the window manager.

Client Contents and Decorations

This is a very infected subject, and it principally requires reaching an agreement  between the clients and the window manager as to the actual contents of client populated buffers. The client cannot do anything to stop the window manager from decorating its windows, and the window manager cannot stop the client from rendering decorations into the buffer used as canvas. The window manager only has a slight advantage as it can forcibly crop the buffer during composition and translate input coordinates accordingly, as shown in this (demo).

Thus, the client need to know if it should, or should not, draw titlebar, border and so on. This decision has far reaching consequences as to the information that needs to be conveyed, understood and synchronised between the window manager and the clients. For instance, titlebar buttons drawn on the client side need to reflect server side window state, and suddenly needs information if it is in a ‘minimised’, ‘maximised’ or some other state.

It is further confused by the part that technically, what many refer to as ‘server side’ decorations are not exactly ‘server side’ – the predominant X solution is that one or even multiple clients (window manager and compositor as possibly separate clients) decorate others. The big problem with this approach, other than it becomes almost uniquely complicated, is synchronisation. The decorations need to reflect the state and size of the decorated client and the speed at which you can do this is limited to the combined round-trip time for the one doing the decorating and the one being decorated.

The principal standpoint in Arcan is that the server is allowed to be complicated to write as it will only be written once, and the clients should be as trivial as possible. The ‘annotations’ that decorations provide should be entirely up to the window manager.

Since the WM is an entirely different and privileged thing to a client here, we can have ‘true’ server side decorations – with the typical approach being to just synthesise them on demand as a shader during the composition stage at a fraction of the cost, even for fancy things like drop shadows.

The ‘compromise’ is that clients can explicitly either indicate the ‘look’ of a cursor from a preset W3C derived list, or request subsegments of a ‘cursor’ and/or a ‘titlebar’ type. As the case is with all secondary allocations these requests are default-deny and window manager opt-in.

Clipboard

The clipboard is traditionally one of the most strange forms of IPC around, with the implementation in Windows arguably being the biggest piece of “WTF” around, tailgated by X. The hallmark “feature” is that you can’t be certain of whatever it is that you “paste” – its origins and its destination, as the interface is inherently prone to race conditions.

The venerable ICCCM defines three ‘selection buffers’: PRIMARY, SECONDARY and CLIPBOARD, with the real difference being the inputs these are bound to (ctrl+c, ctrl+v or menus for CLIPBOARD vs mouse actions for PRIMARY). External clients, clipboard managers, can be used to add / monitor these buffers. There is also the option for clients to negotiate content format by exchanging list of supported formats, and a lengthy protocol for Drag and Drop.

In Arcan, there is no abstract singleton ‘clipboard’ or ‘drag and drop as such’, but rather two types of segments: CLIPBOARD (client to server) and CLIPBOARD_PASTE (server to client). If a client wish to provide clipboard information, it can request a subsegment of the CLIPBOARD type, and if a paste operation occurs, the server force-pushes a CLIPBOARD_PASTE subsegment into the client. These will all accept both audio, video, short messages and file descriptor transfer routing.

This allows for an infinite number of “clipboards”, and how these are shared between different clients is determined by the window manager. Thus, it can be completely configurable and safe to allow one client to act as a clipboard manager or segment into trusted and non-trusted groups for that matter. Example code of a simple clipboard manager can be found here.

It can be set to directly copy one client clipboard with another client clipboard-paste with the window manager intercepting and modifying everything, or to just act as a file descriptor router, you pick your own poison.

Synchronisation

While it might be a bit confusing to think of ‘synchronisation’ as a feature, there are a few things to consider that might make it a bit more clear. Modern system graphics is parallelised and asynchronous to a degree that cannot really be underestimated, and then hits like a brick where you fail to think things through. The task is not made any easier by the fact that most of the parts in the chain that needs synchronising are unreliable, limited or just plain broken.

A fantastic explanation of the absolute basics in this regard can be found in this article by Jasper st. Pierre et. all: XPLAIN. Then follow the developments in this regard to the PRESENT extension and check up on the XSYNC extension. I will not attempt to summarise those here and instead go with a model of the problem itself.

There are three big areas we should consider for synchronisation:

The first area is synchronisation to the displays.

Historically, display synchronisation was ‘easy’ in that you could literally have an interrupt handler that told you when the display wanted new contents and how much time you had left to produce it. This became difficult when you hooked up multiple displays that do not necessarily share the same refresh rate or refresh method.

As an example, when a 60Hz display cooperating with a 50Hz in trying to share contents, you’d likely prefer tearing to the latency of using the least common divisor as the the shared synch period

Xorg tries its hardest to meet the specific deadlines of the displays that are connected. This is a necessity when trying to ‘race the beam’, down to counting microseconds. This was an admirable approach at the time of its conception, and not so much today.

The second area is the window manager.

The window manager needs to react when its UI components are being manipulated. It needs to relayout / redecorate accordingly, and be ready in time for composition and scanout (synch to display). This can be hard to impossible to strike with both accuracy and precision, with many external factors introducing jitter and delays. This is an area where animation quality tend to highlight how well the solution works as a whole, and actually a good reason to have them for troubleshooting, regardless if they are enabled in production or not.

The third area is synchronising to clients themselves.

A common, if not the most common, kind of client is interactive. This means that it mainly updates or produces new contents in response to the input that it receives. Thus the state of the input samples that are forwarded needs to be fresh or latency suffers. At the same time, the client need to decide when to stop waiting for input and when to use that input to synthesise the output. The state itself might need client local coordinate transforms which might depend on surface dimensions. The source device might be a high sample-rate mouse device emitting 1khz of samples to a terminal emulator capable of handling a fraction of that. The list goes on and ends up in a web of judgement calls that balances latency to cpu utilisation.

Now, how does Arcan approach the range of synchronisation problems then? Well, on a number of levels.

High level: There is a runtime changeable ‘strategy’ that can be set to favour some property like input latency; animation smoothness; or energy conservation. This is a user choice that gets applied regardless of what the window manager itself decides.

Mid-level: The window manager has controls to set a ‘synchronisation target’ that act as input to the high level strategy. The currently selected window being a good target. This means that the synchronisation target gets favourable treatment.

Low-level: A single client can be ‘mapped’ to an output, side-stepping window management in its entirety. The window manager can also force a synchronisation mode and even manually clock and suggest or enforce the deadline per client.

API-level: A client has some movement range itself in that it can cancel current synchronisation attempts (at the cost of possible tearing), bind itself to ‘as early as possible’, against a dynamic deadline or a ‘triple-buffer’ like strategy where the latest complete frame is the one that gets synched out.

Phew. Needlessly to say, there is a lot more that can be said on the subject while still barely scratching the surface. The next article in this series, “at feature parity”, will be a while, as the features needed are still not in place. If you have not guessed it by now, that will mainly be about multi-level network transparency and drawing command-stream buffer formats. The last article then, can come in quick succession as the features that will be highlighted are all working or in a near-working state. Features like 3D compositing, VR, audio support, live migration, server recovery, multi-GPU rendering and so on.

If you have made it this far, congratulations. Also, why not join in? Some of the work is a lot more fun than it might seem.

Posted in Uncategorized | 2 Comments

Arcan 0.5.5, Durden 0.5

We’ve accumulated enough features and fixes that it is time for yet another release. The beefier changelogs can be found here: (Arcan) and here: (Durden). Starting with the following video which consolidates the visible changes to (indirectly) Arcan and its reference desktop environment, Durden.

Among the many release highlights is the improved OpenBSD support and that we are now packaged in void linux (wiki entry), the premier Linux distribution for those of us who would much prefer a user space free of the more incessant “freedesktop.org” impositions. With the exception of the recently added ‘Safespaces‘ VR desktop project, and its associated hardware device driver bridges, most other key components and supplementary tools are now packaged and ready to be enjoyed inside the void. The Arcan package has been synched to match 0.5.5 and Durden 0.5 will come in a day or two.

In terms of future steps:

The Arcan 0.5 focus with its focus on graphics related subsystems will likely wind down towards the end of the year and put more attention towards advanced network support (that is, not RDP/VNC nonsense), being the key part we lack before comfortably being able to claim to be way beyond feature parity with Xorg.

Durden has about one or two big release more with a feature focus in it (particularly for touch displays, tablets and styluses), then we will switch over to improving the ones we already have and polish away the ‘developer graphics’ aesthetics and just sand down the edges in general.

After this release, the respective git repositories will be managed a bit differently from the ‘everything goes on master’ approach from before. After each ‘big’ release such as this one, a set of key features per subsystem gets picked from ‘the infinitely big pile’ and gets added as a branch with a corresponding (VERSION-SUBSYS-NOTES) file in the root covering a rough breakdown of the changes to be made before the branch is complete.

When a branch has been completed, it gets cleaned up, rebased, squashed and merged unto master – prompting a new version bump (a.b.c.d+1) until there are no more such branches, prompting a new blog post /video and a (a.b.c+1) versioned release.

Thus, if you are monitoring the git – be forewarned that activity on master will only come in bursts and that the topic branches will be the new way to go.

You can see the current ones here: ARCAN and DURDEN. Now, onto the highlighted feature walkthrough.

Browser Improvement: Video Previews

The built-in command-line interface resource browser has been extended with the option to allow known video formats to be automatically previewed when selected or when they pop into view. More detailed controls over preview triggers have also been added.

Relevant menu paths: /global/settings/browser

The next steps for this feature is to add ‘set actions’ for the filtered set, like open as playlist. We are also investigating letting external clients provide previews (based on priority and level of trust), as well as open to client, letting the browser act as a universal file picker.

New Feature: Menu system is mountable

All UI interaction and configuration can now be accessed ‘as a file’. Combined with the added ‘arcan_cfgfs’ (arcan:src/tools/acfgfs) FUSE file system tool, the entire menu system is now mountable – albeit  still a bit slow:

This opens up for a number of testing / automation features, and should also make it easier to discover all the available features when you can basically find and grep. It is implemented over the ‘control’ domain socket. Enable it with the the menu path:

/global/settings/system/control=i_am_in_control

and a socket with the name ‘i_am_in_control’ will be created in the durden/ipc path in the appl_temp filesystem namespace (depending on how you run it, for a local build it might be the same as /path/to/durden, $XDG_DATA_HOME/arcan/appl-out/durden/ipc or $HOME/.arcan/appl-out/durden/ipc on some distributions). You can also access the socket manually or via other/custom tools (fastest approach) than FUSE:

socat - unix-client:/path/to/i_am_in_control

Here you should be able to issue commands like:

 ls /
read /global/settings/visual/mouse_scale
exec /browse
write /global/settings/visual/mouse_scale=2

This also opens up for runtime conversion / application of configuration files from other WM systems, assisting with workflow migration.

It is also possible to monitor events from various desktop subsystems:

monitor wm notification display timers

(current groups being ‘notification’, ‘wm’, ‘display’, ‘connection’, ‘input’, ‘dispatch’, ‘wayland’, ‘ipc’, ‘timers’, ‘clipboard’, ‘clients’)

Combining the monitor feature with the ‘everything is a file’ approach should be complete enough to allow the conversion of old xdotool- like hacks, or, being brave enough, translate the relevant xlib subsets to provide support for running Xorg WMs.

New Tool: VRViewer

It is no coincidence that this tool looks eerily similar to the VR Desktop (Safespaces) – it is mostly the exact same code base, with small parts changed to integrate into the desktop.

The primary uses for this tool are both for developing and testing safespaces without having to enable/disable a HMD every so often, but also for data viewing use cases where you might have stereoscopic and/or geometry projected videos or photos that you want to look at quickly.

Relevant menu paths: /global/tools/vrviewer

The video also shows “drag-and-drop” like cursor-tagging, integrated into the tool to allow windows to migrate or clone into the VR tool.

Advanced Float improvements:

The ‘advanced float’ mode is something of a more hidden tool that slowly but surely aggregates all the features that your rodent friendly DE would provide. Two new features this time: a reveal / hide mode to the autolayouter, and a Window-to-Wallpaper feature that can forward input when no native window has input focus. Below, you can see an instance of Xarcan being used as the ‘wallpaper’. The gridfit- tool that allows compiz- like pseudo-tiling has also received a popup- mode for a pointer device friendly quickpath.

Relevant menu paths: /target/window/workspace_background

The final coming stretch for this part of Durden should be binding menu paths to background icons, and allow window minimization targets to be statusbar, tray or desktop.

New Feature: Decoration Impostors

As a follow up to the argument on client side decorations, we now allow per-client custom cropping of [t | l | d | r] pixels, and the option to define a titlebar impostor that can be toggled on and off. In the video you see gtk3-demo under wayland getting its incessant border and shadow removed, and the “fattyfingerbar” becoming a custom toggle in order to not lose access to client controls. There are still a number of edge cases to work out, but the basic concept is working.

Relevant menu paths: /target/window/crop, /target/window/titlebar/impostor

Speaking of decorations, it is now possible to bind arbitrary menu paths to titlebar buttons in UI (previous method was only by patching autostart.lua and worked globally), with unique per window overrides or mode-specific sets that follow the current workspace layout mode. Workspaces in float mode can also have a different border thickness and border area than the other modes.

The next steps for this feature is automatic detection, integration with the ‘viewport’ protocol, to allow the selected window titlebar to ‘merge’ into the statusbar center-area, to relayout it to be attached to the sides of the window, and to provide an ‘external impostor’ for the statusbar (which is just a special version of the titlebar, same UI code) via a connection point.

New Widget – notifications:

When various subsystems or clients have anything important to say, the message is added to a queue that will be flushed whenever the HUD is interactively toggled. This also occurs on certain important events, like the error triggering crash recovery or abnormal client termination. You can also attach notifications of your own via the IPC system mentioned earlier.

Relevant menu paths: /global/settings/notifications/* (with send=… to emit)

The video clip showed a test client that shows a single colour window for a brief moment, then shuts down due to some “problem” but just before the connection is severed, the client can attach a short ‘last_words’ message to provide a user-readable message explaining why it terminated abnormally. This is added as a notification.

Think of it as the UI friendly equivalent of program exit codes. Since crash recovery in the WM side has gotten seamless enough that you often don’t even notice it happened, the message also carries over as a notification which is also shown in the video.

 

Posted in Uncategorized | 3 Comments

Revisiting the Arcan Project

Two years has passed since the public presentation where Arcan and its ecosystem of side projects and tools started to creep out of the woodworks; although the project had been alive and worked on for well-over a decade at that point. A lot of things have evolved and developed in a positive direction since the time of that presentation, and this post aims to provide an overview of those changes.

But before that, the ‘elevator- pitch’ to the modern tweet-length attention spanned reader goes as follows:

Arcan is a curious blend of a streaming multimedia processor, game engine and display server with a novel design that lends itself well for both complex and simple realtime interactive graphics projects alike, and goes well with anything from Sci-Fi UIs for some homegrown embedded project to full blown desktops. It is highly modular, low on dependencies, yet comes with all necessary batteries included.

The corresponding pitch for the related project Durden is as follows:

Durden is a desktop environment dedicated to- and designed first and foremost for- hackers and professional users. As such it is keyboard dominant, building a better and more efficient CLI than the flaccid terminal emulators of yore. The strategy is to amass as many relevant features and workflows as possible, then leave them turned off yet within runtime reach – with an ‘everything is a file’ approach to control, configuration and automation.

With these reintroductions in place, let us dig through the major developments to the plot:

Stream processing
Crash Recovery and Crash Resilience
VR support
Plan9/UI Security experiments
LEDs as a first class citizen
TUI: Text-based User Interfaces
X and Wayland support
BSD ports

Stream processing

In order to further stress the point that the processing model, engine design and developer APIs are much more generic and flexible than the mere ‘Display Server’ archetype would permit or warrant, the different client- and output- connection mechanisms have all been extended for easier composition with itself. The simply means that multiple arcan instances can chain into each-other, each with a different set of processing scripts.

This feature makes it possible to interactively (with user input) stream-process multimedia (both audio and video), just like we have been able to do with text for oh so long. In this way, the feature also becomes a building block for more feature rich and dynamic media processing and composition than is provided by the likes of the venerable OBS-Studio. For more on this, see: “AWK” for multimedia.

VR support

One might be tempted to think “supporting VR” is simply stereoscopic rendering with a barrel distortion post-process and camera controls tied to a head attached IMU sensor, but that is just the entry level test.

The real thing, unfortunately, requires careful contents planning, load balancing, camera processing, video overlays, positional audio and so on – managing a virtual zoo of complex input devices that makes the nightmares of touchscreen management look like mere push buttons by comparison.

A lot of the basic controls for this have been added to Arcan and many more are to come during the year. The process was recently covered and exemplified with the post Safespaces: An Open Source VR Desktop. It shows how the building blocks are coming together for usable VR (and 3D) desktop workflows.

Crash Recovery and Crash Resilience

This part alone pushed the timeline back a year or two from the earlier roadmap, but was well worth it. An almost unreasonable amount of work has been done in order to make sure that there is incrementally stronger separation between the user interface (window manager), the display server and its clients, and that each of these should be able to recover if one or even two of the other parts would fail.

Clients can detach when they detect that the display server connection is severed, and enter a sleep-reconnect state or switch to a different display server and rebuild itself there, and for Wayland clients, this is done unbeknownst to the client itself. This is one substantial building block towards getting ‘better than X’ network transparency, an upcoming focus target.

For those that are interested, additional technical detail is presented in the article “Crash-Resilient Wayland Compositing” and this is an area where we are close to surpassing the strongest competitor in the field (Windows).

TUI : Text-based User Interfaces

The lessons learnt from writing the built in terminal emulator led to the start of development towards a new API with a number of advanced goals – the main of which is to show that the terminal protocols are ghastly display server protocols in disguise. The associated article: The Dawn of a new Command Line Interface elaborates on this in greater detail – although it is past due for an update.

The condensed point is that the isolated world that APIs (today, ncurses) and related shell (your CLI window manager, e.g. bash, zsh, …) live in due to the terminal protocol heritage radically lowers the quality of the unix command line and doubly so from the schism it has towards the graphical desktop. Arcan is closing in on a way to bridge that gap.

Plan9/UI Security Experiments

The article ‘One night in Rio – Vacation photos from Plan9′ showed how the design and API could be used to expand on a core interesting feature from the window management scheme used in Plan9. This was achieved in two stages:

First stage was providing hierarchical connection tracking and compartmentation, forcing clients spawned from a specific terminal to be tied to tabs belonging to the same window the terminal has, with no means for breaking free.

Second stage was using the connection point namespace (how an external clients finds the server) to force per-connection unique window management policies to account for ‘special needs’ clients/roles like external wallpapers, ‘rofi’ like overlay hud, statusbar etc. without any modifications to client or API.

On top of that, clients gets universally split into trust domains based on the origins of their connection and identity tokens, selectively allowing rate limiting and different permission sets on external connections and their sub-windows.

LEDs as a first class Citizen

Historically, LED output requirements have been modest from a display server perspective – boiling down to possible indicator lights on keyboards. The current reality is that there are many more outputs than the display today to control for ergonomics and overall user experience.

In the post “Playing with LEDs” covers how the once humble subsystem that dealt with external LED controllers for custom projects has been extended to controls for display backlight, RGB keyboards and mice, keyboard backlight and so on.

X and Wayland Support

Compatibility with various external clients have been treated as a lower priority than all the other tasks for a number of reasons that warrants a lengthier, dedicated post. Two years ago the only real compatibility options were as VMs via a patched QEmu backend.

To make this situation more tolerable, a separate Xorg backend, Xarcan was added after it was deemed that was actually much less work than adding support for XWayland. This is covered in the post “Dating my X“.

Wayland support was also added by taking a cue from the world of microkernels and implementing it as a translation service. This approach turned out quite well as a means of compartmentalising, sandboxing and least-privilege separating each wayland client at the protocol level, and the translation service will likely be extended to shield of / contain the radiation from more of certain “free”-desktop projects.

BSD ports

With some minor build system coaxing and input system modifications, most of Arcan can now be built and run on the various BSDs. The tougher nut to crack in this regard was unsurprisingly OpenBSD, With the article ‘Towards Secure System Graphics: Arcan and OpenBSD‘ showing all the little steps and considerations that had to be taken, and what is left to be done.

Though I would normally like to end a post like this with some tepid visions towards the coming two years and the project future in general, this time I will just retsrain myself and hint that it is time to get considerably more hardcore.

Posted in Uncategorized | 4 Comments

Towards Secure System Graphics: Arcan and OpenBSD

Let me preface this by saying that this is a (very) long and medium-rare technical article about the security considerations and minutiae of porting (most of) the Arcan ecosystem to work under OpenBSD. The main point of this article is not so much flirting with the OpenBSD crowd or adding further noise to software engineering topics, but to go through the special considerations that had to be taken, as notes to anyone else that decides to go down this overgrown and lonesome trail, or are curious about some less than obvious differences between how these things “work” on Linux vs. other parts of the world.

A disclaimer is also that most of this have been discovered by experimentation and combining bits and pieces scattered in everything from Xorg code to man pages, there may be smarter ways to solve some of the problems mentioned – this is just the best I could find within the time allotted. I’d be happy to be corrected, in patch/pull request form that is 😉

Each section will start with a short rant-like explanation of how it works in Linux, and what the translation to OpenBSD involved or, in the cases that are still partly or fully missing, will require. The topics that will be covered this time are:

Motivation

One of the many lofty goals behind Arcan has been to not only reduce system-wide desktop complexity, to push the envelope in terms of system graphics features, quality and performance – but also to enable experimentation with security sensitive workflows and interaction schemes, with the One Night in Rio: Vacation Photos from Plan9 article covering one such experiment. Working off the classic Confidentiality- Integrity- Availability- information security staple, the article on Crash-Resilient Wayland Compositing also showed the effort being put into the Availability aspect.

The bigger picture of how much this actually entails will be saved for a different article, but the running notes on the current state of things, as well as higher level experiments are kept in [Engine Security] and [Durden-Security/Safety].

Outside of attack surface reduction, exploit mitigations and safety features, there is a more mundane, yet important aspect of security – and that is of software quality itself. This is an area where it is easy to succumb to fad language “fanboyism” or to drift in the Software Homeopathy direction of some bastardised form of code autism, such as counting the number of lines of code per function, indentation style, comment:code ratio and many other ridiculous metrics.

While there are many parts to reaching a lofty goal of supposed ‘quality software’, the one that is the implicit target of this post is the value of a portable codebase as a way of weening out bugs (where #ifdef hell or ‘stick it in a VM masked as a language runtime’ does not count).

The road to- and value of- a portable codebase is built on the ability to swap out system integration layers as a way of questioning the existing codebase about hidden assumptions and reliance on non-standard, non-portable and possibly non-robust behaviours and interfaces.

This does not always have to be on the OS level, but in other layers in the stack as well. For instance, in the domain that Arcan targets, the OpenGL implementation can be switched out for one that uses GLES, though their respective feature sets manifest many painful yet subtle differences.

Similarly, the accelerated buffer sharing mechanism can be swapped between DMA-Buf and EGLStreams. Low-level DRI and evdev based access for graphics and input can be swapped out for high-level LibSDL based. Engine builds are tested with multiple libc:s to avoid “glibc-rot” and so on. Some of these cases are not without friction, and even painful at times, but the net contribution of being able to swap out key culprits as a means of finding and resolving bugs have, in my experience, been a positive one.

Arcan has been ported to quite a few platforms by now – even if not all of them are publicly accessible; there are certain ecosystems (e.g. android, iOS) we have very little interest in publicly supporting in a normal way. The latest addition to the list of actually supported platforms is OpenBSD, added about a year after we had support for FreeBSD.  So, enough with the high-level banter, time to dive into the technical meat of the matter.

Graphics Device Access Differences

Recall that in the BSDs, most of the DRM+GBM+Mesa (DRI) stack (or as Ilja van Sprundel poignantly and accurately puts it – the shit sandwich) is mostly lifted straight from Linux, hence why some of the interfaces are decidedly “non-BSD” in their look and feel.

Device Nodes and drmMaster

A normal (e)udev setup provides device nodes on linux as /dev/dri/Card + RenderD. In OpenBSD, these nodes are located at /dev/drm, but merely changing the corresponding ‘open’ calls won’t yield much of a success, unless you are running as root. The reason is a piece of ugly known as drmMaster.

As with most things linux these days, if something previously had permissions – there is now likely both a permission, some hidden additional permission layer and some kind of service manager that shift things around if you ever get too comfortable. Something that once looked like a file actually has the characteristics and behaviour of a file system; file permissions and ownership gets “extended” with hidden attributes, capabilities, and so on.

This is the case here as well. A user may well have R/W access to the device node, but only a user with CAP_SYS_ROOT/ADMIN is allowed to become “drmMaster”, a singleton role that determines who is allowed to configure display outputs (modeset) or send data (scanout) to a display, and who that is allowed to only use the device node to access certain buffers and setup accelerated graphics. As a note, this will get more complicated with the ongoing addition of ‘leases’ – tokenised outsourced slicing of display outputs connectors.

In Linux, there are about 2.5 solutions to this problem. The trivial one is weston-launch, have it installed suid and there you go. The second is an ugly monster that we can call a “YAFDOBF” (Yet Another FreeDesktop.Org Brain Fart). It involves an unholy union between Logind, D-Bus, udev and the ‘session’ and ‘s+[ea | hi]+t’ abstractions that ultimately leads to a permanent state of not being able to trust file descriptors.

The last .5 is the one we have been using up until recently, and involves taking advantage of a corner case in that you do not have to set drmMaster – if no-one else has tried to; buffer scanout and modeset actually work anyway, just ignore the failed call. Set normal user permissions on the card node and there we go. This has the risk that an untrusted process running as the same user can do the same and break display state, which leads us to a case of “avoid mixing trust domains within the same uid/gid”. If you really desire to run software that you don’t trust, well, give such miscreants another uid/gid that has permission on the ‘RenderD’ node for privilege separation and the problem would be solved.

Alas, neither the Master-less scanout nor the render node feature currently exist on OpenBSD,  leaving the no-option of running as root or the remaining one of doing what weston-launch did, but in a saner way. This means taking a cue from Xenocara. Make the binary suid, and the first thing you do, no argument parsing, no hidden __constructor nonsense, just setup a socketpair, fork and drop privileges. Implement a trivial open/close protocol over the socketpair, and have a whitelist of permitted devices.

Since we need dynamic and reactive access to a number of other devices anyhow, this tactic works well enough for that as well. There is a caveat here in that the user running the thing can successfully send kill -KILL to the supposedly root process, breaking certain operations like VT switching (see the Missing section).

Input

One of the absolutely murkiest corner in any user – kernel space interface has to be that of input devices: keyboards, trackpads, joysticks, mice and so on. This is one of the rare few areas where I can sit and skim Android documentation and silently whisper “if only” before going back to alternating between screaming into a pillow and tracing through an ever expanding tree of “works 100% – 95% of the time” translation tables, where each new branch manages to be just a tad bit dumber than the ones that came before it.

Starting in Linux land again, first open the documentation. Now, you can close it again – it won’t be useful – I just wanted you to get a glimpse of its horrors. The only interface that can really be used, is evdev, written in cursive just to indicate that it should be pronounced with a hiss. It is a black lump of code that will never turn into a diamond. Instead of being replaced, it gets an ever increasing pile of “helper” libraries (libevdev, libudev, libinput, libratbag, libkilljoy, libreadabuttonpress, libvaluebetween0and1, …) that tries to hide how shitty this layer is, a decay pattern which matches the DLL creep in Windows – without trying to address the core of the matter. The reason for this pattern is, I assume, that no one in their right mind wants to actually work on such a thing within the Linux kernel ecosystem – if it can at all be avoided.

For OpenBSD, the better source of information is probably the PDF (Input Handling in wscons and X) along with the related manpages on wscons, uhid, wsmouse, wskbd. For the most part, these are obvious and simple, with the caveat that most of the low level controls are left to other tools, or that they work as multiplexer abstract devices without any ‘stream identifier’ for demultiplexation. Then there’s the case of the keyboard layout format. The underlying “problem”, depending on your point of view, is the legacy that exist in both Linux and BSDs in that tradition has meant going from BIOS -> boot loader(s) -> “text” console -> (display manager) -> Xorg and we at least need the option of being able to boot straight into UI.

Hotplug

There are two sides to the hotplug coin: input devices and output devices. The normal developer facing Linux strategy uses udev, the source code of which is recommended reading for anyone still looking for more reasons to get away from linux systems programming. If you want to have extra fun, take a look at how udev distinguishes a keyboard from a joystick.

Udev is an abomination that has been on my blacklist for a number of years. It has all the hallmark traits of a parasitic dependency and is a hard one to ignore. For input devices, the choice was simply to go with inotify on a user provided folder (default, /dev/input), and let whatever automatic or manual process he has populating that folder with the device nodes he thinks that Arcan should try and use.

For output, one might be easily fooled to think that the card device node that is used for everything (almost, see the backlight section) that relate to displays like mode setting, buffer scanout and synch triggers, should also be used to detect when a display is added or removed to this card. Welcome to Linux. How about, instead, we send an event with data over a socket (Netlink) in a comically shitty format, to a daemon (udev) which maps to a path in a filesystem (sysfs best pronounced as Sisyphus) which gets scraped and correlated to a database and then broadcasted to a set of listeners over D-Bus. Oh goodie. One might be so bold to call this an odd design, given that all we need is a signal to trigger a rescan of the CRTCs (“Cathode Ray ConTrollers” for a fun legacy word) as the related API basically forces us to do our own tracking of connector card set deltas anyhow.

In OpenBSD (>= 6.3) you add the file descriptor of the card device node to a kqueue (❤️) set and wait for it to signal.

Backlight

Backlight is typically reserved for the laptop use case where it is an important one, as not only can modern screens be murdering bright, but also bat away at your battery. This subsystem is actually quite involved in Arcan as it maps as yet another LED controller, though one that gets paired to ‘display added’ events. Hence why we can do things like in  this (youtube video).

The approach we used for Linux is lifted from libbacklight before it became a part of the katamari damacy game that is systemd. Again one might be tempted to think that the backlight control could somehow be tied to the display that it regulates the brightness of, but that is not how this works. Just like with hotplug, there’s scraping around in sysfs.

In OpenBSD, you grab the console tty and send an ioctl to it.

Xorg

As touched upon in the section on graphics device access, OpenBSD maintains its own Xorg fork known as Xenocara in order to maintain their specific brand of privilege separation. Arcan meanwhile maintains its own Xarcan fork to get more flexibility and security options than what XWayland can offer, at the cost of legacy application “transparency” (though that will be introduced via the Wayland bridge at some point).

The only real difference here right now, in addition to the much superior DRI3(000) rendering and data passing model is not working (see the Missing section further below) – was adding translation tables to go from SDL1.2 keysyms (the unfortunate format that we have to work with, no one, especially not me, is without sin here).

Pledging

A large part of the work behind Arcan has been to resection the entire desktop side of user space to permit much more fine grained privilege separation, particularly when it comes to parsers working on untrusted data – but also for server to client sharing modes. Thus we split out as much of the data transformations and specialised data providers as possible into their own processes – and the same goes for data parsing.

One of the reasons for this is, of course, that these are error prone tasks that should not be allowed to damage the rest of the system in any way, segmenting these out and monitoring for crashes is a decent passive way of finding new less than exciting vulnerabilities in FFmpeg, something that still happens much too often some ten years later.

The other goal is, of course, to be able to apply more precise sandboxing as one process gets assigned one task, like a good Mr. Meeseeks. An indirect beneficial side effect to this is that we get a context for experimenting with the different sandboxing experiences that the various OSes provide, such as the case of the supplementary tool ‘aloadimage‘ (a quite paranoid version of xloadimage).

Missing

The OpenBSD port of DRI current lacks two important features – render nodes and DMA buffers. This means that we have no sane way of sending accelerated buffers between a client using accelerated graphics and Arcan, and we put little effort in providing support for the older way (GEM/…). This breaks the nested ‘arcan-in-arcan’ mode with the arcan_lwa binary, it breaks wayland support and it breaks Xarcan supporting DRI3 as these all rely on that feature.

While it is technically possibly to explicitly enable old GEM style GPU sharing (durden exposes these as config/system/GPU delegation) where clients to be trusted authenticate via a challenge response scheme to the GPU then gets the keys to the kingdom, it is far from a recommended way as this puts the client on just about the same privilege terms as the display server itself. That being said, it is murderously foolish to think that a client who is allowed any form of raw GPU access these days doesn’t have multiple ways of privilege escalation waiting to be discovered; the order of priority is getting them to do what they are told without crashing the system, then to account for clients that rely on unspecified or buggy behaviour and maybe to get them to perform under these conditions. Security is a distant figure somewhere on the horizon.

Last and incidentally, my least favourite part of the entire ecosystem, “libwayland-server” still lacks a working port – which comes as no surprise as this thread unsafe, typecasting crazy, faux-vtable loving, use-after free factory is littered with no-nos; the least of which being its pointless reliance on ‘epoll‘.

Posted in Uncategorized | 2 Comments

Safespaces: An Open Source VR Desktop

In this post, I will go through the current stages of work on a 3D and (optionally) VR desktop for the Arcan display server. It is tentatively called safespaces (Github link) as an ironic remark on the ‘anything but safe’ state of what is waiting inside. For the impatient, here is a video of it being used (Youtube link):

The explanation of what is going on can be found in the ‘High Level Use’ section further below.

To help navigate the post, here are a few links to the individual sections:

Background and Motivation

One of the absolute main goals with the Arcan project as a whole is to explore different models for all the technical aspects that goes into how we interact with computers, and to provide the infrastructure for reducing the barrier to entry for such explorations.

The overall ambition is that it should be ‘patching- scripts’ levels of difficult to piece together or tune a complete desktop environment to fit your individual fancy, and that major parts of such scripts could – with little to no modification – be shared or reused across projects.

For this reason, I’ve deliberately tried to avoid repeating- or imitating- the designs that have been used by projects such as Windows, Android, Xorg, OS X and so on. The reason was not to question or challenge their technical or business related soundness as such; those merits are already fact. Instead, the reason was to find things that was not exactly “sound” business – hence why this has all been kept as a non-profit, self-financed thing.

During the last few months, I’ve started unlocking engine parts that were intended for building VR environments, and many of the design choices had this idea as part of the implicit requirements specification very early on. The biggest hurdle has been hardware quality and availability, an area where we are finally starting to reach a workable level – the Vive/PSVR- level of hardware is definitely “good enough” to start experimenting – yet terrible enough to not be taken seriously.

Arcan as a whole is in a prime position to do this “right”, in part because of its possible role as both a display server, and as a streaming multimedia processor / aggregator. Some of the details of the TUI subproject and the SHMIF IPC subsystem also fit into the bigger puzzle, as their implicit effect is to push for thinking of clients as interactive, streams of different synchronised data types – rather than opaque pixmaps produced in monolithic ‘GUI toolkit thiefdoms’.

High level use and Demo

Starting with a shorter but slightly more crowded video from the same desktop session:

In the video, you see parts of the first window management scheme that I am trying out. Although the video is presented as monoscopic, the contents itself is stereoscopic – 3D encoded video looks 3D when viewed on a HMD.

The window management scheme is a riff on the tiling window manager, mostly due to the current lack of more sophisticated input devices that would benefit from something more refined. Actual input is keyboard and mouse in this round, though there are ongoing experiments with building a suitable glove.

“Windows”, or rather, models – since you have a number of different shapes and mappings to chose from – are grouped in cylindrical layers. The user is fixed at the centre and the input focused window is positioned at 12 o clock, with other sibling windows rotated to face the user (“billboarding”), scaled down and positioned around the layer geometry.

Each layer can be arranged at either a fixed distance for heads-up display components and infinite geometry like skyboxes; or they can be swapped back and forth – with optional opacity fade-offs, level of detail triggers and so on based on layer distance.

In each layer you can attach various primitive models, from cylinders and spheres to rectangles, flat or curved. Each model can be assigned an external source, an image and an optional activation connection point. Individual properties, such as stereoscopic mapping modes, scale, opacity and so on can also be set, and the models can swap places with each-other within the layer. Each layer can have a ‘layouter’ attached that automatically determines window positions and scale. The builtin default works like in this figure:

layer

This shows a single layer with the user is at a fixed position, dead center. The window that has input focus has the center slot at 12’o clock, and the other windows are evenly laid out around a circle that match the set per-layer radius. If the focus window spawns subwindows or hierarchically bound subconnections, like when a terminal starts a graphics program, such windows gets positioned vertically.

The activation connection point is a unique name where the ARCAN_CONNPATH environment variable points. In the video you can see a designated ‘moviescreen’ appearing at the end when I redirect video there, and disappearing when the specific client disconnects.

A little twist is that safespaces was actually written as a tool (‘vrviewer’) for Durden – even though it is also a full window manager in disguise. The reason why I went this path is for prototyping agility, taking advantage of the tons of existing code and features in Durden shortens the ‘edit, run, test’ cycle drastically. It also eliminates the headache of picking the right display for the right purpose and other low level time consuming details – and you can move your workflows back and forth between the 3D/VR state and the 2D one.

The downside is that there is quite some overhead running it nested like this since Arcan also needs to take the normal desktop management into account, and there is some interference in the refresh rates of the different displays I have hooked up.

Setup and Architecture

There is a tool in the Arcan codebase that you have to build and enable, vrbridge (arcan_vr). It has some weird constraints since we definitely don’t want to do the device management in-process – yet the sampled data has a tiny window of opportunity for use. It takes extended, privileged features in the SHMIF API (so the engine will have to launch and manage the program. Instructions for enabling are in the README.md file). The current version supports devices via OpenHMD, but it is trivial to interface with other VR- related APIs.

The design of the vrbridge interface is such that it allows for selective plugging of a wide range of devices (gloves, haptic suits, eye trackers, …). The scripting layer activates the VR bridge and gets announcements of ‘limbs’ arriving or disappearing. A limb gets activated by the script mapping the limb to a 3D model, and the engine will take care of synchronising limb orientation and so on at both a monotonic rate (for collision detection and response) and when preparing new output frames.

The biggest problem right now – is interfacing with devices and the vast array of possible pluggable input devices that extend the VR setup capabilities. It doesn’t help that practically everyone tries to push their own little lock-in happy frameworks and makes it hard (deliberately or by accident) to access the very few control primitives and sensor samples that are actually needed. As an added bonus, there is also “incentive” to merge with the next round of walled garden app-stores, because things weren’t bad enough as is.

Next Steps

This setup is still in its infancy, and the work so far has highlighted a few sharp corners that will need sanding, although the biggest eyesores (literally) – the quality of the 3D pipeline and the positional audio will have to wait for a while until the relevant stages of the longer Arcan roadmap has been reached.

Another part of that reason is due to the low level of system integration and high level portability requirements that Arcan needs to follow; we are really restricted as to the set of GPU features that can be relied upon and purposely restrictive when it comes to introducing dependencies.

There are two big low level targets for the near- future:

The first is improved asymmetric multi-GPU support. The challenge of this task scales drastically with what you actually do, where sending textured quads back and forth is trivial, and then it goes bad quick and nightmarish almost as fast. The two worst parts is effective load balancing and rewriting much of the synchronisation and storage management code to multithread better and get advantage out of adaptive synchronisation outputs (FreeSync).

The second is fleshing out the interaction with clients so that there is intelligent level-of-detail, and the ability for the client to output 3D representations rather than just rendered pixel buffers – think mesh icons, voxel buffers etc. for server-side rendering in order to get better immersion and seamless, gradual transition / handover between desktop use and dedicated client use.

 

Posted in Uncategorized | 6 Comments

Argumenting Client-Side Decorations

Apparently it is the season to chime in on opinions on client side decorations (CSDs) versus server side decorations (SSDs) in the context of Wayland. I normally ignore these kinds of quarrels but this is a case where I consider the dominating solution being close to the worst possible one. The two pieces of note, so far, is This post from camp Gnome and This from camp KDE.

For starters, Martin claims:

Nothing in Wayland enforces CSD. Wayland itself is as ignorant about this as X11.

Well, it is the compositor’s job to enforce it (i.e. do nothing really) and he doesn’t want to – it’s a case of bad hygiene for any implementor of a protocol to skew interpretation this blatantly as it counteracts the point of having one. You don’t have to dig deep to find the intention behind the protocol – even the first commit to the project has:

Window management is largely pushed to the clients, they
draw their own decorations and move and resize themselves, typically
implemented in a toolkit library.

This has been debated again and again, and if the ‘me-too’:ers would have taken even a cursory google glance at the mailing list rather than skewing things to fit their narrative (or just not doing the basic research because why would you), they would immediately run into Kristians recurring opinion about it, and he designed much of the protocol. If that does not align with your wants and needs, perhaps this is not the protocol you should use, instead of coming up with worse extensions where the client can simply say “no, I want to decorate” and the server have to comply or break protocol. It also doesn’t do anything about the mouse cursor.

Furthermore, a ton of stuff in Wayland make very little sense without CSDs – particularly wl_subsurface and wl_region but also, to some extent, the xdg_shell parts about window state (maximized, movable, fullscreen, resizing, context menu).

Then we have the topic of the #wayland IRC channel itself:

Please do not argue about server-side vs. client side decorations. It's settled and won't change.

That said, I also think that the agreed upon approach to CSDs is a technically inept, and this post will lay out those reasons.

Lets first look at what the decorations chatter is all about, from this screenshot you have the top bar, the shadow region and the window border and, what is not visible on the screenshot – the mouse cursor. All these are possibly client defined UI components.

csds.png

The options to convey this segmentation in wayland is practically as a single unit (a surface) or a complex aggregate that is a major pain to implement, using a part of the protocol that should be taken out behind the shed and executed (a surface with a tree of subsurfaces).

Reading through comments on the matter, most seem to focus on the value of the top bar and how much stuff you can shove into it to save vertical space on a floating window manager with widescreen monitors.

Had that been the only thing, I would have cared less, because well, the strategy that will be used to counter this in the various Arcan related projects is a riff on this:

 

I’ll automatically detect where the top bar is, crop it out and hide it inside a titlebar toggle in the server side defined titlebar where I have the stuff that I want to be able to do to the client. There is practically nothing GTK, or anyone else for that matter, can do to counter that. In durden, the /target/window/crop and /target/window/titlebar/impostor features allows for that, per window.

Protocol wise, it would be nice if the bar that GTK/GNOME (and to be fair, Chrome, Firefox, …) prefer in their design language had been added as another possible surface role to xdg-surface so the segmentation could go a bit smoother and the border and shadow could all be left entirely to the compositor – but it is, all in all, a minor thing.

There are, however, other areas that I argue that the choice matter a lot more, so lets look at those.

Security

If you look like projects such as QubesOS (see also: border color spoofing) and Prio, the border part of the decorations can be used to annotate signal compartment or security domain. The server side annotates each client surface with its knowledge about the contents, domain and trust level that the user should associate with the particular client.

cubes.png

With client side decorations there is, by definition, no contribution from the compositor side of the equation. This means that any client can proxy the look, feel and behaviour of any other.

Even if I have a trusted path of execution where the compositor spawns the specific binary (like Arcan can with a sandboxing chainloader and a policy database) keeping the chain of trust intact. Without a visual way to indicate this, there is no difference from a compromised process spawning and proxying an instance of the same binary as a man in the middle in order to snoop on keyboard and clipboard input and client output. The problem is not insurmountable, but other design language need to be invented and trained.

Latency

Not that there are many serious attempts to allow networking for single wayland clients at the moment, but for networked clients, normal window operations like move and drag resize now require a full round-trip and a buffer swap to indicate that the operation is possible since the mouse  cursor need to be changed to reflect the current valid state. Having such operations jitter in their responsiveness is really awkward to work with.

That said, there are other parts of the protocol that has similar problems, particularly the client being responsible for maintaining the keyboard layout state machine and repeat-rate.

Performance and Memory Consumption

The obvious part is that all buffers need to be larger than necessary in order to account for the shadow and border region. This implies that more data needs to be transferred to- and cached on- the GPU side. About the scarcest resource I have on my systems is memory on GPUs and when it thrashes, it hurts.

Everything the client draws has to be transferred in some way, with the absolute fastest way being one well-aligned buffer that gets sent to the GPU with a handle (descriptor) being returned in kind. This handle gets forwarded to the compositor that maps it to a texture and uses this texture to draw. The crutch here is that there are many ways to draw primitives that use this surface, and the one you chose will have impact on how expensive it will be for the GPU.

The shadow region needs to be drawn with alpha blending enabled or it will be ugly. The vast majority of UI contents can be drawn without any kind of blending enabled, but with client side decorations the compositor is left with no choice. To combat this, there is the wl_region part of the wayland protocol, which, lacking nicer ways to put it, is profoundly dumb. Basically, you annotate the regions of a surface that is to be ‘opaque” so the compositor can perform the lovely dance of slicing up drawing into degenerate quads and toggle the blend states on and off. My memory may be a bit fuzzy, but I do not recall anyone compositor that actually bothers with this.

The position here is that the shadow and border- part of CSDs increase the surface that has to be considered for all drawing, buffering and layout operations. Consequently, they increase the amount of data that has to be updated, transferred and synched. The main content, furthermore, gets pushed to offsets where they don’t align with efficient transfers making the most expensive operation we have (texture uploads) worse. They also mutate the contents of the buffer to have a mixed origin and type. Because of this, they also adversely affect compression and other quality parameters like filtering, mipmapping and antialiasing.

UI Consistency

This is the part many discussion threads on the matter seem to boil down to so I will not provide much annotation on it. The stance that gets repeated from GTK people is a version of “users do not care, Chrome and friends do this anyhow”.  Here you have GTK, QT, EFL, SDL and Weston in loving harmony. Everyone with their own take on colors, icons, buttons and mouse cursor. “Beautiful”.

 

Client and Protocol Complexity

It is considerably more complex to write a conforming wayland client than it is to write an Xorg client. A simple “hello world” press a key and flip a color output is in the range of thousands of lines of code, and the problem of actually knowing how to draw client decorations is open ended, as you get no hints in regards to how the user wants things to look or behave.

This means that some clients just ignore it – to some conflict and chagrin – Wayland SDL and GLFW backends or clients like Retroarch do not implement them at all, for instance. The inflammatory thread in this MPV issue also highlights the problem.

The “I need to draw decorations?” problem becomes worse when the contents the client wants to present is in a format that is hard or costly to draw decorations in because of their rendering pipeline or restrictions on colour formats that should be used, incidentally, the case for games and video playback. This is where the wl_subsurface protocol is needed, and it maps back into the performance perspective.

The tactic is that you split the surface into the main contents area with one buffer format, and draw the decorations in slices using some other. These can be infinitely many and a fun way of making all compositors crash right now if you know what you are doing.

For operations like drag- resize and transforms, these becomes painfully complicated to deal with as you may need to wait for all surfaces in the tree to be “ready” before you can use them again. They can easily consume around ~12 file descriptors even if the client is being nice, and with a few tricks it can become much worse.

It is to no surprise that GTK, EFL and others agree on the current situation, as they have already done the work, thus this empowers them at the cost of everyone else. Pundits also typically chime in and say something like “everyone is using a toolkit anyway” to which I say, step out of your filter bubble and widen your sample set, there are plenty of “raw” clients if you know where to look. The natural follow up from the same crowd is something to the effect of “you shouldn’t write a wayland client manually yourself, use a toolkit” – which is nothing but a terrible excuse and undermining the point of agreeing on a protocol when the details gets masked inside library code, it means moving the complexity around, making it less visible, rather than actually reducing complexity.

Adding a dependency to a million-line level codebase is a really weird way of making something “simple”.  I will keep my less than polite response to this to myself and abuse a quote from the, relatively speaking, more balanced Bill Hicks:

Suck a tail-pipe, fucking hang yourself, borrow a gun from a Yank friend – I don’t care how you do it. Rid the world of your evil fucking machinations.

If the point of wayland is, in fact, the often claimed “make it simpler” (and not say, transfer control over a large part of the linux ecosystem to your organisation), when the suggested solution turns out worse than the dominating solution and your counterargument to those problems is “hide it in a library” – you are being dishonest and deceptive or dumb, it is as simple as that.

Posted in Uncategorized | 1 Comment

Arcan 0.5.4, Durden 0.4

‘Tis the season to be jolly and just about time for the definitely last release of the year for both Arcan and its related subproject, the Durden desktop environment. Following the pattern from the last release post, lets go through some highlights and related videos, but first – a family photo:

IMG_0636

From left to right, we have a little Raspberry  running the ‘prio’ WM using the broadcom binary blob drivers (so lacks some of the features needed to run durden), with arcan and terminals eating up all of 20MB of ram. The left Macbook running OSX with Arcan/Durden in fullscreen, retina resolution, of course. The Macbook on the right is running the same system on OpenBSD 6.2. The three-headed monkey behind them is a voidlinux setup with two instances, one on an intel GPU, the other on an AMD GPU. If only the android devices on the wall could be brought in on the fun as well…

Arcan

For Arcan itself, there has been rather few ‘visible’ changes, but many more things underneath.

The Xorg arcan backend ‘Xarcan’ has been ported to OpenBSD and is now up and running there. While on the subject, the OpenBSD port has also received some improvements on the input front, with support for the wsmouse interface and the engine itself now uses the same privilege separation setup as Xenocara.

The VRbridge tool has been updated with basic OpenHMD support. In the video, you can see the vrtest script where a cube is mapped to the reported HMD orientation. A number of engine improvements have also been made for managing VR related rendering. As a PoC – here’s windowmaker, xterm and xeyes connected to an xarcan instance that renders to arcan_lwa and then via arcan with this script (gist).

 

The wayland protocol service has seen quite a few improvements to crash recovery, wl-shell and xdg-shell protocol compliance. It has also received a single-exec mode for compartmentation of wayland clients, and has some basic least-privileges via seccomp- filters. The full details of the crash recovery steps are kept in a separate article, Crash Resilient Wayland compositing.

The Terminal and TUI libraries have been extended to get a window ‘cloning’ feature and a related pasteboard mode. The clone feature acts as a passive ‘screenshot’ of the contents of the terminal at activation time, and the pasteboard mode reroutes ‘copy’ operations to be added to a copy window rather than being added to the clipboard directly.

Arcan and the special ‘lwa’ build have gotten their argument handling and initial setup extended to allow arbitrarily long chains of ‘pipes and filters’ like media processing, the full details are kept in the separate article, AWK for multimedia.

Yet another tool has joined the ranks, arcan-netproxy, though not really usable for anything but experimentation at the moment, but it will become the main development focus target for the 0.6.x versions. Its role and purpose is to provide ‘per- window’ like networking features, and later on, full network transparency.

Durden

To weigh up for the lack of visual flair to Arcan itself, plenty of highly visible changes has been made to the durden desktop environment.

Flair

Flair is a new tool script that provides a Compiz- like UI effects framework for adding visual flair to normal operations. Though only a few effects have been added so far, more are sure to come now that the tough parts have been finished. In the video you see not only faux-physics ‘clothy windows’ (spring grid, verlet integration, obviously superior to wobbly windows) but also an xsnow- like effect layer and a burn-on-window-destroy effect. Now, if we only could get an adaptation of Realtime 2D Radiosity and end the era of drop shadows…

Overview

Overview is a tool that acts as an ‘expose’ like workspace switcher to more quickly see what is happening on the other workspaces. Though it lacks something in the looks department, the bigger work effort was, like with the flair tool, getting the infrastructure in place for hooking and handling. Later iterations will bring back more interesting workspace switching looks.

Advanced Float

Previously, the floating window management mode was quite crude, and efforts have started to spice things up a bit. It is still not entirely competitive with advanced floating window managers yet, but the missing features should appear around the next version. This round of enhancements adds support for:

  • spawn control (e.g. draw to spawn shown in the video)
  • minimise targets (e.g. minimise to statusbar or desktop-icon)
  • customised “on-drag-enter/over” screen regions
  • titlebar buttons can now be set based on workspace mode
  • auto-rearranger (simple binpack for now)
  • grid- like position/sizing controls (like the compiz plugin)

Here you see the spawn control used, to position/size a number of windows, then auto-rearranging them.

Terminal Group- Mode

Taking a trick from the Rio/Plan9 article, it is now possible to spawn a terminal ‘group’ which act as its own distinct connection point. Graphical clients that connect via this point forcibly shares group (window slot and settings) with the terminal it spawned from, making hybrid text/graphics clients easier to work with in tiling modes. In the video, you see how the terminal switches to an image when aloadimage is run, switching back/forth between the parent slot and children, and back again when the window is ‘destroyed’.

Menu and Browser Improvements

The browser filtering now respects wildcards, but also lua patterns and specialised sort-order controls. In addition, most menu paths now show a description of what each entry do, and navigation also works with the mouse.

What’s Next?

The tedious 0.5.x- lower system graphics interfacing is sure to continue for a while longer, with the next round of changes focusing further on multi-vendor-multi-GPU drawing, swapping and load balancing now that most of the necessary configuration system rework has been dealt with.

Another worthwhile work target will be improving synchronisation strategies now that drivers start to support FreeSync, particularly letting the WM decide if and when some client should be prioritised (think games and latency) to complement the existing ‘direct-to-screen’ mode.

It is also about time to push the work on TUI further: finishing the Lua API bindings and related advanced features e.g. subwindows, content embedding, standalone version (kmscon: the return).

For Durden, the feature race will soon (but not yet) start to slow down and instead focus on raising overall quality and usability (polish, actual default profiles, internationalisation support). Before then, however, it is likely that some more advanced input devices (eye trackers, customised mouse- and touchscreen gestures, onscreen keyboard) will get a little bit of love, icon and drag/drop for float mode, and the model-viewer tool being extended to support redirecting to a VR HMD, allowing for 360-, 180- video playback.

Senseye has been neglected for a long time, with only minor experiments on how to refactor the UI into something better. After some pondering, the individual sensors will be changed into “normal” arcan applications, and all the analysis tools will become plugins to Durden rather than acting as a completely separate UI.

Detailed Changelog

Arcan

Engine:

  • VR support now covers the full path from bridge communicating metadata and limb discovery/loss/map/updates
  • (71939f) -0,-1 pipes and filters input setup added, covered in AWK for Multimedia
  • format-string render functions extended with vid-subimage blit

Lua:

  • New function: define_linktarget – used to create an offscreen render pipeline that is tied to the pipeline of another rendertarget
  • New function: subsystem_reset – used to rebuild subsystems (video only for now) to allow live driver upgrades, active GPU switching and so on – without losing state
  • Updated function: camtag_model, change to allow forcing destination rendertarget
  • Updated function: image_tesselation, expose index access
  • Updated function: render_text, added evid,w,h and Evid,w,h,x1,y1,x2,y2
  • Updated function: random_surface, added additional noise function controls

Terminal/Tui

  • Support for ligatures improved
  • Highlighting/Inverse/Full-Block cursor changed for better visibility
  • Added controls to copy the current window into a new (input label: COPY_WINDOW)
  • Copy Windows can be set to be the primary clipboard receiver (input label: SELECT_TOGGLE)

Platform

  • OpenBSD: added mouse support
  • Egl-Dri: swap-GPU slot added to db- based configuration
  • SDL2: improved keyboard and mouse support

Tools

  • VRbridge – initial support for OpenHMD
  • Xarcan – Ported to OpenBSD
  • Waybridge:
    • fixes to subsurface allocations
    • reworked wl-shell to use (most) of the xdg-shell event mapping
    • -egl-shm argument added (perform shm->dma_buf conversion in bridge to offload server)
    • single exec mode (arcan-wayland -exec /my/bin) for stronger separation between clients
    • add support for rebuilding client (crash recovery and migration)

Durden

Big Items:

  • Display region sharing now supports force-pushing into clients that can handle input segments.
  • target/video/advance/migrate,fallback – send a migrate request to a client, which may prompt a client to jump to a different connection point or display server instance.
  • shader subsystem – added a multi-pass effect format along with some initial effects (gaussian blur, CRT-lottes).
  • tools/advfloat – extended float layout mode capabilities:
    • spawn control (draw2spawn)
    • hide-to-statusbar
    • cursor-action-region (see tools/advfloat/cregion.lua for definition)
    • automatic relayouter
  • tools/overview – added a HUD- like workspace switcher
  • tools/flair – added a visual effects layers and some initial effects
  • terminal-group spawn-mode – allows a connection primitive to be generated per terminal and clients which connect via this group share the same logical window tree slot
  • Tui/terminal clients are now allowed to spawn additional tui subsegments.
  • File browser now expose wild-card matching (asterisk), Lua patterns (%%) and sort-order modification (typing % lists options).
  • retain some window and workspace properties across script errors, crashes and resets
  • menu navigation can now shows a helper description of the currently selected item
  • mode-sensitive titlebar icons – window titlebar icons can now be set to be activated only in certain modes

Minor:

  • Destroying a window in fullscreen mode now returns the workspace   to the last known mode instead of forcing to tile
  • Double-tap input-lock automatically unlocks if the locked window   is closed
  • Double-tap input-lock without a selected windows is now a no-op
  • Float mode border drag sizing, cursorhint and positioning fixes
  • Float mode drag now respects statusbar- imposed boundaries
  • Float mode canvas-drag/resize option for self-decorated clients
  • Improved (less broken) handling for wayland popups and subsurfaces
  • Step-resize keybinding now aligns to window- cell size (terminals)
  • statusbar can now be sized/padded to percentage of display output (config/statusbar/borderpad)
  • statusbar specific configuration moved to (config/statusbar) from   (config/visual/bars/…)
  • statusbar number prefix on known workspaces and statusbar mode button can now be toggled on/off

That’s all for now – Happy New Year and See you in 2018.

Posted in Uncategorized | 3 Comments