Updated 1 February 2014 to add the section on Geary.iterate() below.

At my last job, I was doing C#/.NET development. I prefer open to proprietary platforms, but I will say that C# is a great language to work in, and its runtime library has a lot of solid conveniences. One aspect of C# I got hooked on was its robust collections library with vast sets of functional methods to manipulate them. Technically I’m talking about a subset of LINQ, and I’ll narrow the scope of this post to just the regular method or “fluent” syntax, not the special query syntax.

In my current job, I’m mostly working in Vala, whose syntax is modeled on C#’s. There’s a library called libgee (I assume the name is a jab at GLib) that provides Vala with some collections. Libgee is a great start, but it falls short of the convenience of C#’s collections. I don’t want to imply that C# with LINQ is the only way to do collections, but I believe it’s fair to compare Vala and C# here, since Vala is explicit about being modeled after C#.

Where C# has IEnumerable and a lot of extension methods, libgee has both Gee.Iterable and Gee.Traversable. Gee.Traversable is a more recent addition that adds a few LINQ-like functional methods to its Gee.Iterables… and oddly enough, its Gee.Iterators too: they both implement the Gee.Traversable interface. On the plus side, Gee.Traversable lazily evaluates the results of its operations just like the LINQ methods, but there are two big problems that make it a pain to use:

1. Gee.Traversable only adds a few very basic operations. Since Vala doesn’t support extension methods, you’re stuck reimplementing your own first(), distinct(), min(), etc. everywhere you need to call it. Libgee can’t be faulted for Vala’s shortcomings, but it could acknowledge them and go the extra mile for the sake of convenience and DRY.

2. Gee.Traversable’s methods all return Gee.Iterators, which are cumbersome to work with. It’s awkward to manipulate an object that gives you access to one item at a time from a collection, rather than just looking at the collection itself. For example, you can’t pass a Gee.Iterator to foreach. Gee.Traversable defines a @foreach() method to get around this problem, but passing a lambda that requires an explicit return is awful compared to a simple loop, especially considering how Vala compiles down to C code and what that means for your stack traces. Compare these contrived examples:

// This is all I want to do... but this doesn't work.
foreach (Person p in ids.map<Person>(id => new Person(id)).filter(p => p.is_frobular))
    stdout.printf("%s is a frobular person\n", p.name);

// Instead, I have to do this...
ids.map<Person>(id => new Person(id)).filter(p => p.is_frobular).@foreach(p => {
    stdout.printf("%s is a frobular person\n", p.name);
    return true;
});

It should be clear now why Gee.Iterator also implements Gee.Traversable: so you can chain multiple Gee.Traversable method calls together. I posit that it would be far less awkward for Gee.Traversable’s methods to simply return Gee.Iterables, and let Gee.Iterator just deal with iteration.

Anyway, these issues have been slowly and steadily grating on my nerves the whole time I’ve been working on Geary. Recently, I finally set out to implement a fix. My solution, Geary.Iterable, is viewable in this gist. I’ll discuss it in more depth below, but first some examples:

// This tweaked code from above now works, even if the line is a bit long.
foreach (Person p in
        Geary.traverse<Id>(ids).map<Person>(id => new Person(id)).filter(p => p.is_frobular))
    stdout.printf("%s is a frobular person\n", p.name);


// This is how we used to turn folder paths into a map in Geary...
Gee.HashMap<FolderPath, Geary.Folder> existing_folders
        = new Gee.HashMap<FolderPath, Geary.Folder>();
foreach (Geary.Folder folder in existing_list)
    existing_folders.set(folder.path, folder);

// Now look how nice it looks.
Gee.HashMap<FolderPath, Geary.Folder> existing_folders
        = Geary.traverse<Geary.Folder>(existing_list).to_hash_map<FolderPath>(f => f.path);


// Similarly, look at this old code to find matching Geary composer windows...
Gee.List<ComposerWindow> windows = new Gee.LinkedList<ComposerWindow>();
foreach (ComposerWindow w in composer_windows) {
    if (w.account.information == account)
        windows.add(w);
}

// Look how clean now!
Gee.List<ComposerWindow> windows = Geary.traverse<ComposerWindow>(composer_windows)
        .filter(w => w.account.information == account).to_linked_list();

See this commit for more examples from my initial, “lowest hanging fruit” pass over the Geary code. You should already be able to see how useful this new code is.

API Overview

The bulk of this API is the Geary.Iterable class, a lightweight wrapper around a single Gee.Iterator instance. It implements a similar interface to Gee.Traversable, each method calling into the wrapped Gee.Iterator and returning another Geary.Iterable that wraps the result.

Geary.Iterable also implements a Vala-iterable interface, by which I mean it has a method called iterator() that returns an object with methods next() and get(). That’s all Vala needs to be able to use it in foreach.

This gets around the problems above by 1) providing a single place to implement whatever methods might be needed by Geary that aren’t already in Gee.Traversable, and 2) allowing access to whole collections from results, including being able to iterate over them in the standard Vala way. And it adds a minimum of overhead on top of libgee in the process.

Like C#’s IEnumerable, you’re only allowed one iteration on each Geary.Iterable instance, so there are lots of methods to capture the results in all kinds of libgee collections when you need to do something more than just iterate once.

The Geary.traverse() convenience function lets you easily break out of a Gee.Iterable into a Geary.Iterable. If you need to go the opposite direction, you can use to_gee_iterable(), which returns a simple implementation of Gee.Iterable out of a Geary.Iterable.

Update: I later added Geary.iterate() and Geary.iterate_array() as convenience functions to convert a static list or an array into a Geary.Iterable. For example, since Vala lacks C#’s collection initializer syntax, this is the easiest way that I’ve found to make a quick single-item Gee.List:

Geary.iterate<string>("Just one thing...").to_array_list();

It’s been a few weeks since this landed in Geary, and so far I’ve been quite pleased with how useful it is. I haven’t had time to implement much new code to take full advantage of it, but it’s already come in handy enough that I kick myself for not doing this sooner. You also may have noticed that I didn’t implement the full set of LINQ methods in it. More will come with time, as I need them.

The only downside that I’ve encountered using it is more a fault of Vala’s: there’s no generic parameter inference in Vala, so all the necessary angle brackets make things more verbose than I’d like. Take this example from Geary’s source:

return Geary.traverse<Geary.RFC822.MessageID>(ancestors)
    .map_nonnull<Conversation>(a => logical_message_id_map.get(a))
    .to_hash_set();

If Vala was better about inferring generic types, that might start to look like a simple one-liner. Regardless, I still highly recommend implementing something like this in your own Vala projects.