I have been away from wayland system for a while because of work, but I still remember back in the day the pain to manage the window frame (as known as decorations) in wayland system. Surprisingly quite a lot of server work shifted to client side to manage, eg, you need to implement key repeat event in the client applications.
There are two ways right now to do the client decorations.
Through Server side decoration
The wlroots gangs proposed an protocol xdg-decoration to have the server to draw that for you, it is used by fair amount of applications. It’s a bit of against the point of the wayland principles because the compositor is not supposed to draw any thing, it just composes the windows together for archiving best performance. If the server does the frame drawing, it need to either blit clients’ committed surface into sub-region its own copy of the surface, it consumes more memory and it will be bandwidth intensive. Alternatively, the server has to draw the frame in a separated shader, that will consume more GPU cycles.
Up to today, wlroots supports it. Gnome’s mutter is not supporting it, weston projection is not supporting it either.
Client handles it all
The clients handle the situation all, you can use the toolkit like gtk that draws the window frame for you. Or you need to draw the window frame first, in a wl_surface, typically through xdg_surface, then you draw the actual content of your window in the sub-area. Alternatively, you can create a wl_subsurface to set it relative position to the frame, then draw the actual content inside it. The problem is that the frame surface can be fairly big which takes quite amount of memory that is empty(alpha=0). Another problem is that the tiling compositor may not interpret your intention correctly.
The xwayland project, which implements as a wayland client as wayland server, is strongly impacted by it.
libdecor
Today I would like to review an client side solution that is on the second category, it comes to rescuer when you do not have xdg-decoration and no toolkit to help you either. The project is interesting in the sense it is independent of toolkit, you work directly on wl_surface level as opposed to force you to use Qt or gtk. At the same it hides all the decoration window surface for you. There is no documentation on this project explain it how it works nor does the code is evident enough, The only way to use it correctly is to read through the source code. The experiences was the same when I started on the wayland journey 🤦.
libdecor operates on the xdg_surface, when you call libdecor_decorate() on the wl_surface, it will underneath create a xdg_surface for you, which later you can get it through libdecor_frame_get_xdg_surface. This surface is the content surface you are working on, every time you get configure event, you need create a new libdecor_state and commit it using libdecor_frame_commit, typically in this way:
void configure(struct libdecor_frame *frame,
struct libdecor_configuration *configuration)
{
libdecor_configuration_get_window_state(configuration, &window_state);
libdecor_configuration_get_content_size(configuration, frame, &width,
&height);
this->content_width = width;
this->content_height = height;
/* working with libdecor_state */
state = libdecor_state_new(width, height);
libdecor_frame_commit(frame, state, configuration);
libdecor_state_free(state);
/* store floating dimensions */
if (libdecor_frame_is_floating(this->frame)) {
this->floating_width = width;
this->floating_height = height;
}
/* optionally you need to redraw it or if EGL, redraw all the time */
this->redraw();
}
What libdecor_frame_commit does underneath is that it creates several border wl_surfaces as wl_subsurface to the wl_surface you create, because it only draws the title bar. And then of course it would have to implement listeners like wl_subcompositor, and wl_surface.enter/leave/commit, etc. Then you can combine with xdg_shell_surface.set_window_geometry to indicate we have a border here.
static void
frame_set_window_geometry(struct libdecor_frame *frame,
int32_t content_width, int32_t content_height)
{
struct libdecor_plugin *plugin = frame->priv->context->plugin;
int x, y, width, height;
int left, right, top, bottom;
plugin->priv->iface->frame_get_border_size(plugin, frame, NULL,
&left, &right, &top, &bottom);
x = -left;
y = -top;
width = content_width + left + right;
height = content_height + top + bottom;
xdg_surface_set_window_geometry(frame->priv->xdg_surface, x, y, width, height);
}
Note that it is still not fool proof because it indicates negative x,y for the boarder, it requires the compositor to handle negative position, though most compositors I know do handle it.