Safe C wrappers in Rust - Part 1

In this post I want to highlight some aspects of handling wrapping C APIs in Rust that I encountered while working on libzmq bindings. First, I will give some background information, and then grab a specific part of the C API, and discuss how it is mapped in Rust.

Introduction

libzmq is the canonical implementation of the ZMTP protocol, which provides reliable message-passing abstractions on top of TCP, PGM, IPC (i.e. Unix-domain sockets) and in-process, shared memory queues.

While being implemented in C++, it exposes its API in C, as that allows for a stable ABI across compilers and compiler versions on a given platform (which is not possible to achieve using C++).

Having a C API makes the creation of a low-level, unsafe binding in Rust quite straightforward. Such a low-level binding is provided by the zmq-sys crate, used internally by zmq and hosted in the same git repository. However, zmq-sys does not add anything in terms of safety or convenience, it just mirrors the C API in Rust.

So, we can regard zmq-sys as an implementation detail; the interesting stuff happens in the zmq crate — it adds safety, convenience, and tries to expose an idiomatic, "Rusty" API on top of the functionality offered by libzmq.

A few things that can, need, and should be done in creating a safe and idiomatic binding are quite obvious, such as:

  • Introduce Rust wrapper types around all logical types exposed by the C API. Some of these types may be even represented as a void * in C. In Rust, we need strong typing to be able to provide a safe interface.

  • Convert C functions that are logically methods or constructors into actual methods and associated functions in Rust.

  • Convert error return codes into a Error type, and have all functions that might fail return a Result type.

  • Some logical types may not be as obvious, such that a certain group of bit flags belonging together. For these cases, a new type should be created as well, to allow the compiler to catch combinations of flags that do not make sense. The bitflags macro from the bitflags crate comes in handy to do that. However, some flags may benefit from a less a direct translation, such as providing two different methods which different signatures in Rust. In libzmq, it turned out that the flags that specify non-blocking behavior and multi-part messages are not as straightforward to handle as one might think. I hope to discuss the this in a future post.

In summary libzmq poses several interesting challenges in providing a safe and convenient Rust API on top of it:

  • The zmq_getsockopt and zmq_setsockopt functions are tricky in that they are dynamically typed: depending on a runtime function argument, they will return, or accept, a different type. This is the part of the API I want to discuss in this post.

  • There is a thread-safe "context" data type, but the sockets created from it are not thread safe — up to libzmq 4.2. In libzmq 4.2, there are two new socket variants (ZMQ_CLIENT and ZMQ_SERVER) that are thread-safe. Note that new features from libzmq 4.2 are not yet supported by zmq, see issue #122.

  • C does not really have a concept of mutability; const is not transitive, can be circumvented, and is not generally applied consistently in C APIs.

    For the Rust wrapper, we want to capture the actual mutability behavior for types where mutability can be observed. For example, a zmq_msg_t, which is basically a handle for an array of bytes, can be mutated from C, which means it needs to expose its mutating methods taking a &mut self in Rust.

    In other cases, it is not as clear-cut; zmq socket objects are especially interesting. And even for types like zmq_msg_t, which seem straight-forward at the first glance, it might be beneficial to introduce a distinction on the type level. This would also be a worthwhile subject for a future post.

Handling dynamic typing

The API under discussion here is the socket option interface, which will look quite familiar to anyone who has used a BSD-style socket API:

int zmq_setsockopt (void *socket, int option_name,
                    const void *option_value, size_t option_len);
int zmq_getsockopt (void *socket, int option_name,
                    void *option_value, size_t *option_len);

The option_name can be chosen from an assortment of option-naming identifiers evaluating to an integer option ID, e.g. ZMQ_IPV6 or ZMQ_LINGER. Not all options are valid for all sockets, some options may only be applicable for setting, while others may only be read.

But most importantly, the type of the memory passed for option_value and option_len will depend on the option ID. This interface cannot be mapped directly into Rust in a type-safe way.

One solution is to make the dynamic typing of the C interface explicit, and introduce sum type (i.e. Rust enum) to represent the different possible types of values, something like:

enum OptionId {
    ZMQ_IPV6,
    ZMQ_LINGER,
    ...
}
enum OptionValue {
    IntOption(i32),
    BoolOption(bool),
    StringOption(String),
    ...
}

Now our Rust code method signature, and an invocation may look like this:

impl Socket {
    fn set_option(&self, option: SocketId, value: OptionValue) { ... }
    ...
}
...
socket.set_option(zmq::IPV6, BoolOption(true));

Not too bad, but this still has some drawbacks:

  • There is now an additional runtime cost compared to C, as we need to destructure the option value in set_option. This might not be very relevant, as most options are changed or inspected infrequently, if at all. However, it would still be nice to adhere to the "zero-cost abstractions" ideal here.

  • We need to wrap option values into their own type. This could be solved using generics and the Into trait:

    fn set_option<T>(&self, option: SocketOption, value: T)
    where T: Into<OptionValue> { ... }

    This would allow leaving out the BoolOption wrapper:

    socket.set_option(zmq::IPV6, true);

  • There is no compile-time relationship between the option's name and its expected value. This is the sticking point: ideally we would like to ensure the type of value matches the one required by the option used.

The approach currently chosen by the zmq crate is to create separate functions for each option, exposing the matching type, so we get:

fn set_ipv6(&self, bool) { ... }
...
socket.set_ipv6(true);

This ticks off all three points mentioned above, as the type is statically known. However, we now have lost the ability to abstract over option values. For instance, while we could express "a list of socket options" with Vec<(OptionId, OptionValue)> with the first approach, this is not possible with the current approach. I intend to fix this eventually (issue #134), and may write another post when the solution has taken shape.