Folly is an open-source C++ library developed and used at Facebook.
Faux-Folly is a fork of Folly that removes everything that is not required by the Futures library in Folly. It was created by the Nest Camera Software team for use in Camera products.
Folly includes a state-of-the-art C++ Futures/Promises library that is thread-friendly and does not impose the use of coroutines. This opens up better, less-error-prone programming styles when doing asynchronous programming. I won't go into them in this doc, but instead refer you to the Nest presentation “TDD-lite, Futures, and Promises:”
When porting Folly to Brillo/Android, we ran in to numerous problems:
1. The only thing we really want (or need) from Folly is the futures library. Folly is a very broad, boost-like library full of Good Ideas and Bad Ideas and Different Ideas. Developers will constantly try to use the more dubious constructs in the library (e.g. their spin-lock implementation)... and we discourage that. 2. Brillo/Android uses BoringSSL instead of OpenSSL, and we found it too hard to port the code. BoringSSL is a Google fork of OpenSSL, and is neither ABI nor API compatible with OpenSSL. However, it uses the <openssl/*> header namespace and outputs libraries libssl.so and libcrypt.so. In other words, they intentionally make it hard for anyone to have BoringSSL and OpenSSL co-existing on the system. Since we don't have no plans to use the SSL-parts of the library, it makes more sense to remove it. 3. Lots of dependencies. The original dependency list of Folly is a little long, and most of those things have not been ported to Android/Brillo, and most of them are not related to the Futures library. Why ship them on device?? 4. We keep running into problems with Folly because we're violating the assumption that this is an x86_64 server application. In one case, the CacheLocality class unloaded linux-vdso.so.1 because it couldn't find an x86-specific symbol. This breaks things like clock_gettime. Since our team will need to support this library with little or no interaction with Facebook, it needs a smaller surface area to support.
And because of these, we chose to fork the library and trim away everything that wasn't part of the futures library. All said and done, the fork removes over 115,000 LOC.
So... why not use boost::future<> or std::future<>?
std::future<>: as of C++14, there is still no support for asynchronous programming styles/workflows. Namely: they do not support continuations. See https://channel9.msdn.com/Shows/Going+Deep/C-and-Beyond-2012-Herb-Sutter-Concurrency-and-Parallelism . Folly Futures supports this.
boost::future<>: The latest boost thread library does indeed include support for asynchronous programming styles (continuations), but it the destructor of boost::future<> will call wait() if the future is not realized. Put another way: you can not orphan (forget) a future. I.e. you can't do this:
void Foo::bar(SomeObject& o)
{
boost::future<int> f = o.get_value();
f.then([=](int val) {
scoped_lock lock(this->m_mutex);
this->m_val = val;
});
}
The continuation (the .then() lambda) has everything. There's no need to keep the future, we just need to make sure that the continuation fires.
So, with boost this means that you must create some kind of garbage-collection mechanism that will hold all of the unrealized futures and periodically clean them up as futures are realized. That's a lot of bookkeeping, and the purpose of using futures<> is to /reduce/ explicit bookkeeping. Folly Futures /does/ support this style, though it has a few gaps for doing it “right.”
What are the gaps in Folly Futures for orphaning the future?
1. There is no concept of a compulsary cancellation. Once I set the continuation... you must ensure that all references stay valid until the continuation fires. Again... more bookkeeping. However, it's fairly simple to add a cancellation concept to the library. 2. If the future is set to an exception state (i.e. an error), you must either provide an .onError() callback every time you create a .then() callback, or you will not get a notification of the error. This one is trickier to solve without breaking "normal" (non-orphaning) futures use-cases, but we have found a solution where you can designate an executor on which the exception should fire.
So... why not write our own Futures library? Doesn't Folly do a lot of strange things with templates and move semantics that are hard to understand?
As we worked with several libraries that implement futures and promises, we found that writing a high-quality, high-reliability futures library requires a lot of effort and thought to design it well and implement it well. All of the implementations do strange tricks with templates and move semantics. All of the implementations, too, have solution to corner-cases like promise and shared futures. They also include libraries for aggregating futures (e.g. functions that take a list of futures and returns a new future that is realized when any or all of the futures in the list have a value or error). Some of the code we‘ve found that is hard to understand was added because there was a valid use-case that wasn’t working or because there was a deadlock or race condition.
And if we were to write our own, it would end up having almost the same API and implementation as Folly... because it is designed well.
The basic process was something along the lines of:
Once that was done, there were a few libraries that we found to be tangled up:
experimental/fibers: a coroutines library experimental/io: a whole bunch of socket programming stuff
So, we made two (possibly dubious) replacements so that we could break this dependency:
1. Replaced folly::fibers::Baton<> with folly::Baton<>. This isn't a very clean replacement. 2. Remove folly::RequestContext from Futures. This is some kind of system to attach void*-like stuff to future/promises. It came out cleanly. 3. The implementation of Futures was using folly/io/async/HWWheelTimer. In order to remove the async library we needed a replacement. So I wrote a class to give equivalent functionality (but not using the wheel timer algorithm).
New components also come with new unit tests.
It is possible to pull changes from upstream. We forked folly at version 0.52.0. Up until 0.57.0 merges are fairly straightforward and work like this:
0. Start by compiling for your host machine... not Android. 1. Do a git merge on the upstream tag. You will get conflicts. 2. If it is a file we have deleted, keep it deleted. The `git status' will say something like "deleted in ours." 3. If it is a new file in a library we have deleted (experimental/* wangle/*, gen/*, io/async/*) then delete it. 4. If it is a Makefile, you will need to figure out what is added on "our" branch vs. what is added on "their" branch. Usually this is as simple as ignoring the upstream change if they simply added files that we delete. 5. For any other kind of conflict, you'll need to put your engineer's hat on and work it out. 6. Build the code. Ideally, there should be no warnings. 7. Run all the unit tests. They should all pass. 8. As much as possible do only the minimum needed in order to get the code to compile and tests to pass. If you run into a large problem... it is better to break up the changes. Get the merge commit just so that it builds, and then resolve issues on subsequent commits on a mini-branch.
In version 0.57.0, we run in to trouble because of our replacement of Baton<> in Futures doesn't work as expected for some new test cases.
We might rename the .so-file so that this library can co-exist with folly on the same system.
There's probably no need to rename the folly:: namespace.