The Faux Folly Manifest: Intent and Implementation

Folly is an open-source C++ library developed and used at Facebook.

Faux-Folly is a fork of Folly that removes everything that is not required by the Futures library in Folly. It was created by the Nest Camera Software team for use in Camera products.


Folly includes a state-of-the-art C++ Futures/Promises library that is thread-friendly and does not impose the use of coroutines. This opens up better, less-error-prone programming styles when doing asynchronous programming. I won't go into them in this doc, but instead refer you to the Nest presentation “TDD-lite, Futures, and Promises:”

When porting Folly to Brillo/Android, we ran in to numerous problems:

1. The only thing we really want (or need) from Folly is the
   futures library. Folly is a very broad, boost-like library full
   of Good Ideas and Bad Ideas and Different Ideas. Developers
   will constantly try to use the more dubious constructs in the
   library (e.g. their spin-lock implementation)... and we
   discourage that.

2. Brillo/Android uses BoringSSL instead of OpenSSL, and we found
   it too hard to port the code. BoringSSL is a Google fork of
   OpenSSL, and is neither ABI nor API compatible with
   OpenSSL. However, it uses the <openssl/*> header namespace and
   outputs libraries and In other words,
   they intentionally make it hard for anyone to have BoringSSL
   and OpenSSL co-existing on the system. Since we don't have no
   plans to use the SSL-parts of the library, it makes more sense
   to remove it.

3. Lots of dependencies. The original dependency list of Folly is
   a little long, and most of those things have not been ported to
   Android/Brillo, and most of them are not related to the Futures
   library. Why ship them on device??

4. We keep running into problems with Folly because we're
   violating the assumption that this is an x86_64 server
   application. In one case, the CacheLocality class unloaded because it couldn't find an x86-specific
   symbol. This breaks things like clock_gettime. Since our team
   will need to support this library with little or no interaction
   with Facebook, it needs a smaller surface area to support.

And because of these, we chose to fork the library and trim away everything that wasn't part of the futures library. All said and done, the fork removes over 115,000 LOC.

So... why not use boost::future<> or std::future<>?

std::future<>: as of C++14, there is still no support for asynchronous programming styles/workflows. Namely: they do not support continuations. See . Folly Futures supports this.

boost::future<>: The latest boost thread library does indeed include support for asynchronous programming styles (continuations), but it the destructor of boost::future<> will call wait() if the future is not realized. Put another way: you can not orphan (forget) a future. I.e. you can't do this:

void Foo::bar(SomeObject& o)
    boost::future<int> f = o.get_value();
    f.then([=](int val) {
        scoped_lock lock(this->m_mutex);
        this->m_val = val;

The continuation (the .then() lambda) has everything. There's no need to keep the future, we just need to make sure that the continuation fires.

So, with boost this means that you must create some kind of garbage-collection mechanism that will hold all of the unrealized futures and periodically clean them up as futures are realized. That's a lot of bookkeeping, and the purpose of using futures<> is to /reduce/ explicit bookkeeping. Folly Futures /does/ support this style, though it has a few gaps for doing it “right.”

What are the gaps in Folly Futures for orphaning the future?

1. There is no concept of a compulsary cancellation.  Once I set
   the continuation... you must ensure that all references stay
   valid until the continuation fires.  Again... more
   bookkeeping. However, it's fairly simple to add a cancellation
   concept to the library.

2. If the future is set to an exception state (i.e. an error), you
   must either provide an .onError() callback every time you
   create a .then() callback, or you will not get a notification
   of the error. This one is trickier to solve without breaking
   "normal" (non-orphaning) futures use-cases, but we have found a
   solution where you can designate an executor on which the
   exception should fire.

So... why not write our own Futures library? Doesn't Folly do a lot of strange things with templates and move semantics that are hard to understand?

As we worked with several libraries that implement futures and promises, we found that writing a high-quality, high-reliability futures library requires a lot of effort and thought to design it well and implement it well. All of the implementations do strange tricks with templates and move semantics. All of the implementations, too, have solution to corner-cases like promise and shared futures. They also include libraries for aggregating futures (e.g. functions that take a list of futures and returns a new future that is realized when any or all of the futures in the list have a value or error). Some of the code we‘ve found that is hard to understand was added because there was a valid use-case that wasn’t working or because there was a deadlock or race condition.

And if we were to write our own, it would end up having almost the same API and implementation as Folly... because it is designed well.


The basic process was something along the lines of:

  1. Delete something not in folly/futures/ directory.
  2. Compile.
  3. If compile succeeds: run unit tests.
  4. If unit tests pass: commit!
  5. If compile or tests fail: skip. (don't delete it)

Once that was done, there were a few libraries that we found to be tangled up:

experimental/fibers: a coroutines library experimental/io: a whole bunch of socket programming stuff

So, we made two (possibly dubious) replacements so that we could break this dependency:

1. Replaced folly::fibers::Baton<> with folly::Baton<>. This isn't
   a very clean replacement.
2. Remove folly::RequestContext from Futures. This is some kind of
   system to attach void*-like stuff to future/promises. It came
   out cleanly.

3. The implementation of Futures was using folly/io/async/HWWheelTimer.
   In order to remove the async library we needed a replacement. So I
   wrote a class to give equivalent functionality (but not using the
   wheel timer algorithm).

New components also come with new unit tests.

Pulling Changes from Upstream

It is possible to pull changes from upstream. We forked folly at version 0.52.0. Up until 0.57.0 merges are fairly straightforward and work like this:

0. Start by compiling for your host machine... not Android.

1. Do a git merge on the upstream tag. You will get conflicts.

2. If it is a file we have deleted, keep it deleted. The `git
   status' will say something like "deleted in ours."

3. If it is a new file in a library we have deleted
   (experimental/* wangle/*, gen/*, io/async/*) then delete it.

4. If it is a Makefile, you will need to figure out what is
   added on "our" branch vs. what is added on "their" branch.
   Usually this is as simple as ignoring the upstream change
   if they simply added files that we delete.

5. For any other kind of conflict, you'll need to put your
   engineer's hat on and work it out.

6. Build the code. Ideally, there should be no warnings.

7. Run all the unit tests. They should all pass.

8. As much as possible do only the minimum needed in order to
   get the code to compile and tests to pass. If you run into
   a large problem... it is better to break up the changes.
   Get the merge commit just so that it builds, and then resolve
   issues on subsequent commits on a mini-branch.

In version 0.57.0, we run in to trouble because of our replacement of Baton<> in Futures doesn't work as expected for some new test cases.

What's Next?

We might rename the .so-file so that this library can co-exist with folly on the same system.

There's probably no need to rename the folly:: namespace.