This is a follow up article to the higher level writing a kmscon/console replacement. In this article, we will instead use the low-level C API to write a simple client.
To recap, there are 3 APIs for writing clients:
- Advanced/Low- level: “shmif”
- Mid- level: TUI (text-dominant, basic-multimedia)
- High-level: ALT (interactive/advanced multimedia, think ‘app’)
We will cover each of these in turn over several articles, starting with the first one, shmif.
SHMIF or “SHared Memory InterFace” is intended for extending the engine with features like new input drivers; specialised tools that are supposed to extend the window manager you are running; for tight multi-process integration with the engine scene graph and all of its features. The wayland protocol implementation is built on this API, as are things like the libretro support, the QEmu and Xorg backends and so on. In short: it is not for every day. Technically speaking, it is comparable to xlib/xcb/wayland-client, but with quite a few more features, and a lot more comfortable to actually use.
As with the WM article, we will use the ‘console’ git at: https://github.com/letoram/console to store the code.
Prelude
For this we will need a C compiler like gcc or clang, and a build system that can deal with pkgconfig. This time around we will use the meson build system.
Create a meson.build file and an empty main.c file in a folder. Define the meson contents as:
project('demo', 'c', default_options : ['c_std=c11'])
shmif = dependency('arcan-shmif')
executable('demo', 'main.c', dependencies : shmif)
Note the standard being set to C11. Arcan-shmif makes quite heavy use of C11 features and its more advanced memory model, going back to C99 or even older is not an option.
Making a Connection
Open the empty main.c file and add:
#include <arcan_shmif.h>
int main(int argc, char** argv)
{
return EXIT_SUCCESS;
}
An unorthodox detail here is that the shmif headers, by default, pulls in the other headers that it needs from the standard library. The rationale being that it is just a frustrating and annoying time waste remembering all the <stdatomic.h>, <stdint.h>, <inttypes.h> etc. used. There is an ifdef to get rid of that behaviour should one be so religiously inclined.
struct arg_arr* args;
struct arcan_shmif_cont conn =
arcan_shmif_open(SEGID_MEDIA,
SHMIF_ACQUIRE_FATALFAIL, &args);
This is the simplified version of opening a connection, there is also a shmif_open_ext function that allows you to specify a UUID (for settings persistence), application title, current identity and other metadata.
Three things of note here. The first is SEGID_MEDIA, which is the lose type that we identify as. This is primarily a guide for the WM to decide a suitable ‘policy’ in terms of graphics post processing, scheduling and so on.
The second is SHMIF_ACQUIRE_FATALFAIL. There are a number of connection control flags that regulate connection looping, crash recovery, guard threads and other advanced features. The FATALFAIL one here says to simply exit if a connection could not be made.
The third thing is the args parameter (which could be NULL). This act as a way of letting the user or server pass arguments to the application without interfering with the standard mandated argv.
We can also use a slightly more verbose setup:
struct arg_arr* args;
struct arcan_shmif_cont conn = arcan_shmif_open_ext(
SHMIF_ACQUIRE_FATALFAIL, &args, (struct shmif_open_ext){
.type = SEGID_MEDIA, .title = "demo"
}, sizeof struct shmif_open_ext
);
This version allows more information to be provided on connection startup, particularly static / fixed connection title, and dynamic content identity, ident, along with a 128-bit GUID that can be used to help the server side maintain settings persistence across connections.
Inside shmif_cont
This is a technical explanation that can be skipped. If you are not interested, jump forward to the section called ‘Signalling Data’.
The basis of shmif is as the name implies, shared memory. When you make a connection, you get a ‘primary segment’, each connection has one. Destroy it (arcan_shmif_drop) and all other resources are terminated.
The following sketch shows how such a segment is organised:
The context (shmif-cont) is the developer facing structure that contains a local copy of current negotiated properties, like dimensions and buffer sizes. It also has pointers that map to the currently active buffers. Some operations, like arcan_shmif_signal and arcan_shmif_resize may modify this structure and renegotiate the contents of the shared memory.
There is a control socket, a semaphore for audio, video and events that are used to synchronise with the server side. These are primarily there for optimisation purposes, descriptor passing and I/O multiplexing and could be omitted in favour of better memory- based OS primitives (futex + named GPU buffers) where available.
The shared memory is split into two regions, one fixed- size and one variable- sized. The fixed sized region contains negotiation metadata on renegotation, transfer direction (server to client or client to server) as well as a ring buffer for input events, and one for output events. With this setup, events can be routed from their source zero-copy, as well as allowing both sides to have visibility into queue saturation, allowing optimisations like reordering and merging.
The variable-sized region can change on renegotiation (arcan_shmif_resize), where the typically pattern is a change to ‘aproto’ and ‘audio’ after connection setup, then video changes dynamically in response to user actions. Thus for most cases, this can be cheaply extended with truncate+remap.
Aproto contains various more advanced substructures, and are only used for very specialised targets like VR an HDR metadata. Audio works either with multiple-small buffers that the server side initialises to the ideal size of the playback device for latency, or as a big slush buffer for non-latency sensitive streaming audio. Lastly, the video buffer is something we will get into later in the ‘Data Signalling’ section.
Getting Parameters
This is a side-track that can be omitted in our short example, but it is still relevant to bring up. If you recall the ‘writing a WM’ article, there was this weird connection state marked as ‘preroll’, where the WM gets a synchronous step of telling the client all the things it should need to know in order to produce a frame that is immediately useful. This cuts down on tons of connection negotiation verbiage, and greatly reduces the risk of WM policy immediately invalidating the first buffers being produced.
By adding:
struct arcan_shmif_initial* cfg;
arcan_shmif_initial(&conn, &cfg);
We get access to any and all such data that the window manager provided, or some build time default or fallback. This includes properties like:
- accelerated GPU device (resource-handle)
- preferred font(s) (resource-handle) and respective size
- preferred output dimensions, density, colour channel orientation
- audio samplerate
- language, locale and colour scheme preferences
These are soft hints. The client has no obligation to use them, though it would be foolish not to. The related resources will be automatically freed on the next call into shmif.
All of these properties can be changed dynamically as part of the normal event loop, and it is possible for the client to forego the preroll- stage with a flag to the _open call, and thus gain a few ms connection setup time. For this example, since we are not drawing internationalised shaped text, composing UI elements or playing back audio, we can safely ignore all this.
Data Signalling
Time to send some data. In this example we will just draw a gradient that moves to easily see frame, present size and updates. The outer loop will look like this:
size_t ts = 0;
for(;;){
draw_frame(&conn, ts++);
arcan_shmif_signal(&conn, SHMIF_SIGVID);
}
This will block until the server side has acknowledged the transfer and released the client to allow it to produce another frame.
Then the drawing routine something like this:
void draw_frame(struct arcan_shmif_cont* C, size_t ts)
{
float ss = 255.0 / C->w;
float st = 255.0 / C->h;
for (size_t y = 0; y < C->h; y++){
for (size_t x = 0; x < C->w; x++){
uint8_t r = ts + ss * x;
uint8_t g = ts + st * y;
size_t pos = y * C->pitch + x;
C->vidp[pos] = SHMIF_RGBA(r, g, 0, 0xff);
}
}
}
The more eye catching thing about this whole affair should be the shmif_pixel vidp[pos] = SHMIF_RGBA part. In the same way computers have a native endian and so on, this display system has a native compile-time defined pixel format (and audio sample format for that matter). While it can be changed for specialised builds, the vast majority of the time this will be some permutation of 32-bit RGBA, and we statically commit to one and use a packing macro to help clients. The reason is simply to have a safe, guaranteed, default, and leave the more compact- or more complex- formats to advanced transfer mechanisms, like GPU opaque buffer handles.
Now there is easily a ten page essay hidden in unpacking the cases where the render-loop above would be tolerable, and where you would want something more refined, but that is best left for another time; the real subject matter there, synchronisation, is the single most complex system graphics topic around – one that even trumps the subjects of colour spaces and mixed capability output compositing.
Other available options over the same interfaces include:
- Switching to a GPU accelerated context and pass opaque handles
- Toggle alpha channel and linear-/ non-linear RGB colour space
- Dirty-rectangles updates
- Request upcoming deadline
- Set a desired presentation time
- Enable poll()-able event-, timer- or ‘no-synch’ triggered transfers
- Switch to mail-box n-buffers transfers
This would also work with SHMIF_SIGAUD for audio transfers, and both operations could be going on in its own thread with another being responsible for managing the event loop, which is next on the list.
Event Loop
So far our communication has been quite one-sided, how about we replace the frame counter with something event driven, such as when the user presses a certain key, we advance the timer and submit a frame.
Lets get rid of the for (;;) entirely and move to an event loop.
static void event_loop(struct arcan_shmif_cont* C)
{
struct arcan_event ev;
size_t step = 0;
while(arcan_shmif_wait(C, &ev)){
bool dirty = false;
if (ev.category == EVENT_IO)
dirty |= handle_input(C, &ev);
else if (ev.category == EVENT_TARGET)
dirty |= handle_target(C, &ev);
if (!dirty)
continue;
draw_frame(conn, step++);
arcan_shmif_signal(conn, SIGVID);
}
}
This will loop until the parent decides to close the connection. As mentioned before, the event model for engine internals and shmif are actually shared, even though most of the categories are masked out for IPC purposes. The three relevant ones are IO (input devices), TARGET (server->client) and EXTERNAL (client -> server).
We will stub handle target for now:
static bool handle_target(
struct arcan_shmif_cont* C, struct arcan_event* ev)
{
return false;
}
While for input, we set the return to indicate that a new frame should be generated.
static bool handle_input(
struct arcan_shmif_cont* C, struct arcan_event* ev)
{
return (
ev->io.kind == EVENT_IO_BUTTON &&
ev->io.devkind == EVENT_IDEVKIND_KEYBOARD &&
ev->io.datatype == EVENT_IDATATYPE_TRANSLATED &&
ev->io.input.translated.keysym == 108 &&
ev->io.input.translated.active);
}
The IO event structure itself is a big lady that has evolved since practically the start of this project back in ~2003. It encompasses game devices, generic sensors, keyboards, mice, touch screens and eye trackers so far.
Here we are simply concerned if a key is pressed on any keyboard, which has a button like behaviour and translation tables, and if that key resolves to the ‘L’ key (108). That number is actually defined in a static table, inside arcan_tuisym.h and in the builtin/keyboard.lua script.
Just as with synchronisation, there is a lot more that could, and should, be elaborated on regarding problems with this kind of event processing, but small incremental steps is the recipe here.
Renegotiation (Resize)
Time to handle at least one TARGET event, and that is DISPLAYHINT. This event correlates to the WM calling the target_displayhint() function, providing updates about on how the segment will be displayed. This carries density, dimensions, visibility and so on.
So in the previously stubbed handle_target function:
static bool handle_target(
struct arcan_shmif_cont* C, struct arcan_event* ev)
{
switch(ev->tgt.kind){
case TARGET_COMMAND_DISPLAYHINT:{
size_t w = ev->tgt.ioevs[0].uiv;
size_t h = ev->tgt.ioevs[1].uiv;
if (w && h){
arcan_shmif_resize(C, w, h);
return true;
}
}
break;
default:
break;
}
return false;
}
This structure has a more “syscall”- like approach with a small number of primitive data type slots described in the header close to where each event itself is defined. This deviates from both _IO and _EXTERNAL which has actual C union/struct style fields. The reality of that design is mostly legacy, and one that could be adjusted to union- like overloads, but it is low on the list of priorities.
That aside, there is never any direct harm done in not reacting to these events, the server side always assumes minimal capabilities as client default behaviour.
For the event handler here, we simply sanity-check that we get an updated width/height in the DISPLAYHINT, and forward it to the resize operation on the connection. While most operations are asynchronous across the API barrier, resize is one of the exceptions. Partly because it is the most costly and cross-cutting action there is, and partly because so much data / state can change meaning across a resize that it is a reasonable synchronous barrier to have.
Do note that the resize action here does not trigger a resized event in the perspective of the WM. As long as the client stays within allocation tolerances, the synchronous resize is just between the engine’s view of memory mappings and that of the client. The event is actually generated when the newly resized buffer is signalled, and that is true if running across a network as well.
In Closing
This is a good place to stop until next time. If we combine this with the ‘Console’ WM from the last article, we are already at a stage where most of the old school “I want a framebuffer” style applications can be implemented. When we return to this particular API in the series later, we can start to explore the large space of things to do from here, like:
- Spawning new windows
- Defining a custom icon, mouse cursor, titlebar
- Switching to 3D rendered or accelerated handle passing mode
- Handling state block transfers
- Registering custom key bindings, key/value config persistence
- Announcing format encoding/decoding support for universal file picking / browser integration
- Playing back audio
But none of these require anything fundamentally different from what has been covered, merely extending the local event loop handler and emitting a few events.
Arcan 