This post will talk about the difficulties I had when implementing part of a simple but non-trivial D-Bus API using the Emacs bindings.
A few weeks ago we had to do some work related to D-Bus, the IPC solution from FreeDesktop. I took the chance to understand it better, reading a little bit about it and doing some experimenting.
There are some very useful tools to debug and experiment with D-Bus: I used a lot of dbus-send and qdbus (part of Qt library) to call methods of existing objects. Also used d-feet to explore the object interfaces. To monitor interactions in the bus, I used dbus-monitor.
However, one of the APIs we were interested in was the Notification Item, which came from KDE but it is now in the path to become a freedesktop blessed specification. It aims to replace the system tray mechanism based on XEmbed.
The Notification Item is a little bit more complex to play with because it was based on the interaction of different services. You have a daemon to track items and hosts, and hosts will ask items about their properties. So, it wasn’t a simple matter of sending messages, it was necessary to get a service up with certain properties to see it work, i.e. just dbus-send and other tools were not enough.
Even though Qt has very nice tools for binding and implementing D-Bus APIs (for the real work we used those), I was aiming for something more “dynamic”, in the sense that I could build a notification item part by part, and see the interactions.
And then I ended up writing a small, but non-trivial, example of D-Bus usage in Emacs, using the brand new bindings available in Emacs 23. It implements parts of the Notification Item. In a KDE desktop this means you can make an icon appear in the notification area on the panel and interact with Emacs through that icon.
The code is available here, I would be really glad to hear about different ways of doing things. In particular the usage of lexical-let.
The documentation of D-Bus bindings for Emacs is very nice, so I recommend it. Another take on understanding how D-Bus work is looking at KDE’s tutorials, they also explain how to use Qt D-Bus bindings in C++.
I will not go into a line by line detailed description of the code. Instead, I’ll comment on things that I think made the example non-trivial: the usage of lexical-let to create handler (callback) functions, defining D-Bus properties and unregistering a service created in Emacs.
Note that this post is not meant to downplay the Emacs bindings, but to point out my difficulties in a real use, as well as how I overcame them. It really rocked to have those bindings available, without them this post wouldn’t exist and would lose a chance to learn a lot of things about both D-Bus and Emacs Lisp.
The lexical-let
In Emacs, to react to methods being called on a certain object in a service, you have to use the function
(dbus-register-method BUS SERVICE PATH INTERFACE METHOD HANDLER)
so that when that method is called on your object in the given service, the function handler will be called.
In the Notification Item, one of such methods that items have to react to is Activate. In KDE, items are usually represented by icons in the panel, and the action of clicking triggers Activate. There is a SecondaryActivate, which in KDE is triggered by the middle click.
When coding the example, I made the choice that, instead of creating one Notification Item object for the entire session, it would be possible to create different items, with different handlers. And while each method has two integer parameters describing a position where the icon was activated, I wanted to have only one handler function with one parameter describing whether the activation was primary or secondary.
My make-notification-item function takes, among other things, the handler function I described. The code for actually registering the methods is (without the docstring)
(defun ni-register-activate-handlers (service activate)
(lexical-let ((act activate))
(list (dbus-register-method :session service "/NotificationItem"
"org.kde.NotificationItem" "Activate"
(lambda (x y) (funcall act t)))
(dbus-register-method :session service "/NotificationItem"
"org.kde.NotificationItem" "SecondaryActivate"
(lambda (x y) (funcall act nil))))))
so I actually registered a lambda, to filter the discarded parameters and add the primary/secondary parameter. Note that this is going to call the function act, which is not defined in the lambda’s scope.
In many programming languages, what would happen is that the context where act is defined would be saved and attached to the lambda that is being passed around. This feature is what people call lexical binding (or lexical scoping), and the “lambda plus context” is called a closure.
But by default Emacs Lisp doesn’t support ‘closures’ but instead will look into (what I will call) the global context for the act variable. So in my example, I could just use a (setq 'act ...) to change what action would take place when some item was activated. This is called dynamic binding (or dynamic binding). Actually what happens is that when the handler is called, act is not defined, so we get an error.
The code works because of the lexical-let. What it does, roughly binding. Behind the scenes, it is a macro that creates a unique symbol for act in what I called the global context and uses it in the code. So it works in dynamic binding mode. If we somehow found out the name of that generated symbol, we could mess with it just like we could if we were not using lexical-let.
While dynamic scoping is less common, it has some benefits. For a more detailed discussion on that topic, see the Emacs Wiki page DynamicBindingVsLexicalBinding.
The lexical-let is also used when defining the handler functions
that respond to D-Bus queries on properties of the item object.
It seems to me that lexical-let was essential to allow multiple items to be created at the same time. This happens because of the way the bindings are done. The handler function is called without passing any information regarding what D-Bus service/object was called. And at register time we have no other way to pass additional information that would be sent as argument when calling the handler.
If either of those were different, I could have a dispatch table mapping objects to real handlers, or some arbitrary data to real handlers.
D-Bus Properties
While writing the example, I discovered that D-Bus itself has a very low-level mechanism of message passing (or method calling) and many other features are built on top of that.
One of these high level concepts are the D-Bus Properties. In practice the idea is very simple, and could have appeared in different APIs based on D-Bus (maybe that’s the reason they are there…). Instead of having a series of methods for getting data associated with an object, you could have only one, and use an identifier to know which data you are talking about.
For example, in Notification Item, items have among other stuff: a category, a title and a status. Instead of having three getter methods, what the specification of Notification Item does is say that an item have those three properties. Of course, there’s a well defined interface for properties (it is the org.freedesktop.DBus.Properties).
However these high level features are not always available in the bindings. As of the time of this post, the Emacs binding only support reading other objects properties, but not registering new ones. So if you want to do that, you have to implement by hand org.freedesktop.DBus.Properties for your object.
In my code, that’s implemented in my-dbus-register-properties and its auxiliary functions. And that’s another place I’ve used lexical-let, this time to create different handlers for the Get method (part of D-Bus properties interface)
(defun my-dbus-register-get (bus service object prophash)
(lexical-let ((ph prophash))
(dbus-register-method
bus service object dbus-interface-properties "Get"
(lambda (iname pname)
(list `(:variant ,(gethash pname ph)))))))
in the code, I stored the properties of an item in a hash.
How to make a service disappear?
One last issue that I’ve tackled was the fact that the service continued to be visible to D-Bus clients after I unregistered all the methods.
After some research, I’ve found out that what really was happening was that when registering a method for a named service for the first time, the binding requested this name to the server by sending a message to the server object (org.freedesktop.DBus) this is inside the C code of the binding, and uses a wrapper function instead of explicitly building and sending a message.
Inspecting the binding code, I didn’t found the equivalent code for releasing this name in the server. So, even with no methods registered, the named still existed and pointed to the Emacs process.
This was an issue for me because the Notification Item API specifies that to unregister an item, you have to unregister (release) the associated name.
While the solution was simple enough
(defun my-dbus-release-name (service)
(dbus-call-method
:session dbus-service-dbus dbus-path-dbus dbus-interface-dbus
"ReleaseName" service))
this is something I think could be part of the Emacs bindings. Either for doing this automatically (as the requesting names is), or at least to error check when someone is trying to release a name which still have methods registered.
Final thought
It was rewarding to spend time understanding both D-Bus and Elisp. The knowledge helped me understand the D-Bus APIs that we were working at the time, including the Notification Item.
I hope that someone could pick up where I left and try to actually implement a feature in Emacs taking advantage of Notification Item. One obvious feature would be some alert system for the IRC clients in Emacs.