In this post I want to highlight some aspects of handling wrapping C APIs in Rust that I encountered while working on libzmq bindings. First, I will give some background information, and then grab a specific part of the C API, and discuss how it is mapped in Rust.


libzmq is the canonical implementation of the ZMTP protocol, which provides reliable message-passing abstractions on top of TCP, PGM, IPC (i.e. Unix-domain sockets) and in-process, shared memory queues.

While being implemented in C++, it exposes its API in C, as that allows for a stable ABI across compilers and compiler versions on a given platform (which is not possible to achieve using C++).

Having a C API makes the creation of a low-level, unsafe binding in Rust quite straightforward. Such a low-level binding is provided by the zmq-sys crate, used internally by zmq and hosted in the same git repository. However, zmq-sys does not add anything in terms of safety or convenience, it just mirrors the C API in Rust.

So, we can regard zmq-sys as an implementation detail; the interesting stuff happens in the zmq crate — it adds safety, convenience, and tries to expose an idiomatic, "Rusty" API on top of the functionality offered by libzmq.

A few things that can, need, and should be done in creating a safe and idiomatic binding are quite obvious, such as:

  • Introduce Rust wrapper types around all logical types exposed by the C API. Some of these types may be even represented as a void * in C. In Rust, we need strong typing to be able to provide a safe interface.

  • Convert C functions that are logically methods or constructors into actual methods and associated functions in Rust.

  • Convert error return codes into a Error type, and have all functions that might fail return a Result type.

  • Some logical types may not be as obvious, such that a certain group of bit flags belonging together. For these cases, a new type should be created as well, to allow the compiler to catch combinations of flags that do not make sense. The bitflags macro from the bitflags crate comes in handy to do that. However, some flags may benefit from a less a direct translation, such as providing two different methods which different signatures in Rust. In libzmq, it turned out that the flags that specify non-blocking behavior and multi-part messages are not as straightforward to handle as one might think. I hope to discuss the this in a future post.

In summary libzmq poses several interesting challenges in providing a safe and convenient Rust API on top of it:

  • The zmq_getsockopt and zmq_setsockopt functions are tricky in that they are dynamically typed: depending on a runtime function argument, they will return, or accept, a different type. This is the part of the API I want to discuss in this post.

  • There is a thread-safe "context" data type, but the sockets created from it are not thread safe — up to libzmq 4.2. In libzmq 4.2, there are two new socket variants (ZMQ_CLIENT and ZMQ_SERVER) that are thread-safe. Note that new features from libzmq 4.2 are not yet supported by zmq, see issue #122.

  • C does not really have a concept of mutability; const is not transitive, can be circumvented, and is not generally applied consistently in C APIs.

    For the Rust wrapper, we want to capture the actual mutability behavior for types where mutability can be observed. For example, a zmq_msg_t, which is basically a handle for an array of bytes, can be mutated from C, which means it needs to expose its mutating methods taking a &mut self in Rust.

    In other cases, it is not as clear-cut; zmq socket objects are especially interesting. And even for types like zmq_msg_t, which seem straight-forward at the first glance, it might be beneficial to introduce a distinction on the type level. This would also be a worthwhile subject for a future post.

Handling dynamic typing

The API under discussion here is the socket option interface, which will look quite familiar to anyone who has used a BSD-style socket API:

int zmq_setsockopt (void *socket, int option_name,
                    const void *option_value, size_t option_len);
int zmq_getsockopt (void *socket, int option_name,
                    void *option_value, size_t *option_len);

The option_name can be chosen from an assortment of option-naming identifiers evaluating to an integer option ID, e.g. ZMQ_IPV6 or ZMQ_LINGER. Not all options are valid for all sockets, some options may only be applicable for setting, while others may only be read.

But most importantly, the type of the memory passed for option_value and option_len will depend on the option ID. This interface cannot be mapped directly into Rust in a type-safe way.

One solution is to make the dynamic typing of the C interface explicit, and introduce sum type (i.e. Rust enum) to represent the different possible types of values, something like:

enum OptionId {
enum OptionValue {

Now our Rust code method signature, and an invocation may look like this:

impl Socket {
    fn set_option(&self, option: SocketId, value: OptionValue) { ... }
socket.set_option(zmq::IPV6, BoolOption(true));

Not too bad, but this still has some drawbacks:

  • There is now an additional runtime cost compared to C, as we need to destructure the option value in set_option. This might not be very relevant, as most options are changed or inspected infrequently, if at all. However, it would still be nice to adhere to the "zero-cost abstractions" ideal here.

  • We need to wrap option values into their own type. This could be solved using generics and the Into trait:

      fn set_option<T>(&self, option: SocketOption, value: T)
          where T: Into<OptionValue> { ... }

    This would allow leaving out the BoolOption wrapper:

      socket.set_option(zmq::IPV6, true);
  • There is no compile-time relationship between the option's name and its expected value. This is the sticking point: ideally we would like to ensure the type of value matches the one required by the option used.

The approach currently chosen by the zmq crate is to create separate functions for each option, exposing the matching type, so we get:

fn set_ipv6(&self, bool) { ... }

This ticks off all three points mentioned above, as the type is statically known. However, we now have lost the ability to abstract over option values. For instance, while we could express "a list of socket options" with Vec<(OptionId, OptionValue)> with the first approach, this is not possible with the current approach. I intend to fix this eventually (issue #134), and may write another post when the solution has taken shape.

Posted Mon Jan 2 13:07:09 2017 Tags:

Just in case you've wondered how to get Debian's dwww, or other CGI programs, working with the nginx httpd, the following did the trick on Debian 8 (Jessie) for me:

  • Install the fcgiwrap package
  • Add the following location snippet to /etc/sites-available/default:

      server {
        location ~ ^/cgi-bin/([a-z-]+)(\?.*)?$ {
          root /usr/lib/cgi-bin;
          fastcgi_pass   unix:/var/run/fcgiwrap.socket;
          fastcgi_param SCRIPT_FILENAME  $document_root/$1;
          fastcgi_param PATH_INFO $uri;
          fastcgi_param QUERY_INFO $uri;
          # Fastcgi parameters, include the standard ones
          include /etc/nginx/fastcgi_params;

Note that this configuration makes all CGI programs in /usr/lib/cgi-bin available via nginx; this makes perfect sense for a desktop machine, but on a server, you might want to lock things down a bit more.

Posted Wed Jul 22 21:44:16 2015 Tags:

I've now for a long time (6 years it seems) happily used the vcsh script, originally scraped from madduck's zsh setup to manage my configuration files. You can read more about this and other approaches of managing configuration files at the vcs-home website.

To simplify the task of creating a new repository, I wrote, back in those days, a little companion script, which initializes a new repository and sets up the remote based on a command line parameters. As it happens, in the 6 years since, the idea behind the vcsh script has been turned into a featureful tool, packaged for Debian, so I now put my script into the dustbin and made the switch.

However, automatically setting up the remote for newly-created repositories seemed not to be part of the core functionality of vcsh. After studying the man page for a bit, I was delighted to see vcsh came with a flexible hook system. It didn't take me long to come up with a post-init hook, that added that functionality and now even initializes the remote repository. In case you can make use of that functionality yourself, I'll shortly explain the configuration settings releavnt to the hook:

export VCSH_GITURL=<your-host>:<your-directory>

The setting VCSH_GITURL specifies the base directory on the remote host below all of your vcsh git repositories reside. This needs to be a "URL" as understood by by git remote, and will be extended with the name of the vcsh repository being initialized. If set, the hook will use this setting to configure the git remote you'll push your vcsh repository to.

export VCSH_GITHOST=<your-hostname>

VCSH_GITHOST keeps the name of an host reachable via SSH, and will ususally be the host part of VCSH_GITURL.

export VCSH_GITHOST_DIRECTORY=<your-directory>

The directory on VCSH_GITHOST that will keep all of your vcsh git repositories. Usually, this will be the directory part of VCSH_GITURL. If both VCSH_GITHOST and VCSH_GITHOST_DIRECTORY are set, the hook will create a bare git repository on VCSH_GITHOST below VCSH_GITHOST_DIRECTORY.


When a new repository is created according to VCSH_GITHOST and VCSH_GITHOST_DIRECTORY, this setting will be used for generating the description for the newly-created repository. The contents of the file description inside the bare repository is used by e.g. cgit to display a human-readable blurb for the git repository. Note that the contents of VCSH_GITHOST_DESCRIPTION is eval-d by the shell as if it had been enclosed in double quotes, so you can use it as simple template, evaluating other environment variables, as shown above.

Posted Tue Jun 16 00:24:22 2015 Tags:

sbank Tutorial

There's now a tutorial for sbank; this is work-in-progress, but I'd be happy about any suggestions on how to expand it and what to focus on.

New IRClogs features

Also, I've been brushing up my IRClogs webapp recently; it now features:

  • Showing a configurable amount of context around search hits, so you can make more sense of search results without having to go back to the page for that day.

  • Matching phrases in the search results are now highlighted.

  • Each search result entry now links directly to the corresponding day-page entry.

See this link for an example that shows off the features listed above. On the non-user-visible side, there's now support for gzipped logs, which saves a considerable amount of space, just gzip -9 the older log files — this should also work while the IRClogs server is running.

Ruby babbling

I'm using feed2imap to get all those RSS and Atom feeds into my beloved MUA. As I run spam filtering, and split incoming email into Maildir folders on a small server at home (in a setup involving SpamAssassin, Postfix and scmail) and serve those folders over IMAP (using Dovecot), I've decided to add Maildir support to feed2imap, so I no longer have to place my IMAP password in feed2imap's plain-text configuration file.

As feed2imap is implemented in Ruby, this has provided me the opportunity to get my feet wet (or rather moist) in this language. Having a good grip on Python and Scheme, my one-word summary so far would be: Unsurprising!.

If you're interested in the patch, have a watch on the bug report — I'll attach it there once it's in a reasonable shape.

Posted Mon Jul 15 18:48:15 2013 Tags:

While I had a WordPress-based blog quite some time ago (which I rarely made use of), I was never quite content with it; I wanted something more simple, something that does not need a whole LAMP stack, doesn't require tinkering with mod_rewrite, and ideally has a codebase that I'd want to hack on (which definitely rules out WordPress, which is written in PHP).

Tekuti in R6RS clothing

Well, meet Tekuti. It's a lean blogging engine using Git as storage (so you get hot-backup and versioning for free), and, best of all, it's written in Scheme). Since I already have a Scheme HTTP server running for IRClogs, I decided to hack Tekuti so it could run in the same process; this involved porting Tekuti to R6RS and adding code to interface with the web-server (the original Tekuti only provides its services via the mod_lisp protocol). I must say that working with the Tekuti code was a joy; after getting it running using the mod_lisp protocol under Ikarus and Ypsilon, I could add the HTTP integration without touching the rest of the code at all. Kudos to Andy Wingo for writing this nice piece of code.

Yet another Scheme Web server

To avoid code duplication, I re-factored the ad-hoc libsoup-based web server that drove the IRClogs application into its own library. The interface provided by the library is very much inspired by the SUnet Web Server and hides libsoup entirely.

So, how does my script for running IRClogs and Tekuti inside the SPEnet HTTPd look like? Well, leaving out the customization and library imports, like this:

(define (rforge-request-handler irclogs)
   `(("blog" . ,(tekuti-handler))
     ("irclogs" . ,(lambda (path request)
                     (irclogs 'dispatch path request))))
   (rooted-file-handler *static-dir*)))

(let ((irclogs (make-irclogs *irclogs-config*)))
   (httpd (make-httpd-options
           with-port *port*
           with-request-handler (rforge-request-handler irclogs))
          (lambda (httpd)
            (irclogs 'update-state)
            (httpd/add-timeout httpd 60 (lambda ()
                                          (irclogs 'update-state)

Running the above code, with port, static-dir and irclogs-config appropriatly defined, will launch a completely stand-alone web server that will serve IRC logs, run a Tekuti blog instance, and provide static content from static-dir.


You can fetch the R6RS port of Tekuti via git:

git clone git://
git checkout t/r6rs/do-port
Posted Mon Jul 15 18:48:15 2013 Tags: