| # Thread safe asynchronous code | |
| Writing correct asynchronous programs with multiple threads requires care in | |
| C++. Here we describe a particular pattern that helps avoid errors, and which | |
| will integrate well with the C++ FIDL bindings and component runtime. | |
| ## Background | |
| ### Asynchronous runtimes | |
| The [async][async-readme] library defines the *interface* | |
| for initiating asynchronous operations on Fuchsia. It defines an opaque | |
| `async_dispatcher_t` type, and associated functions. | |
| There are several *implementations* of this dispatcher interface. A popular one | |
| is [`async_loop_t`][async-loop] and its C++ wrapper | |
| [`async::Loop`][async-loop-cpp]. Libraries that performs asynchronous work | |
| generally should not know what is the concrete implementation. Instead they | |
| would call functions over the `async_dispatcher_t*` interface. | |
| ### Thread safety | |
| The reader should familiarize themselves with the terminology around | |
| [thread safety][thread-safety] if needed. | |
| A program that upholds thread safety avoids data races: broadly, reading and | |
| writing the same data without a defined ordering between those operations (see | |
| precise definition of a [data race][data-race] in the C++ standard). These races | |
| are a source of errors because they lead to undefined behavior at run-time. | |
| An individual C++ type also has categorizations around thread-safety. Referring | |
| common practice interpretations from [abseil][abseil-thread-safety]: | |
| - A C++ object is *thread-safe* if concurrent usages does not cause data races. | |
| - A C++ object is *thread-unsafe* if any concurrent usage may cause data races. | |
| One may wrap a thread-unsafe type with *synchronization primitives* e.g. mutexes | |
| to make it thread-safe. This is called adding *external synchronization*. Doing | |
| so adds overhead, and not all users will use that type concurrently. Hence it's | |
| common for a library to be thread-unsafe by default, and require the user to add | |
| synchronization if desired. Such types may have comments like the following: | |
| ```c++ | |
| // This class is thread-unsafe. Methods require external synchronization. | |
| class SomeUnsafeType { /* ... */ }; | |
| ``` | |
| ## Achieving thread safety in asynchronous code | |
| Achieving thread safety gets more subtle in asynchronous code due to the | |
| presence of callbacks. Consider the following snippet: | |
| ```c++ | |
| // |CsvParser| asynchronously reads from a file, and parses the contents as | |
| // comma separated values. | |
| class CsvParser{ | |
| public: | |
| void Load() { | |
| reader_.AsyncRead([this] (std::string data) { | |
| values_ = Parse(data); | |
| }); | |
| } | |
| std::vector<std::string> Parse(const std::string& data); | |
| private: | |
| FileReader reader_; | |
| std::vector<std::string> values_; | |
| }; | |
| ``` | |
| `AsyncRead` will complete the work in the background, then call the lambda | |
| specified as the callback function when the work completes. Because the lambda | |
| captures `this`, it is commonly referred to as an "upcall": the `reader_` that | |
| is owned by an instance of `CsvParser` makes a call to the owner. | |
| Let's consider how to avoid races between this callback and the destruction of | |
| `CsvParser`. Adding a mutex in `CsvParser` won't help, because the mutex would | |
| be destroyed if `CsvParser` is destroyed. One may require that `CsvParser` must | |
| always be reference counted, but that results in an opinionated API and tends to | |
| recursively cause everything referenced by `CsvParser` to also be reference | |
| counted. | |
| If we ensure that there is always a defined ordering between the destruction of | |
| `CsvParser` and the invocation of the callback, then the race condition is | |
| avoided. On Fuchsia, the callback is typically scheduled on an | |
| `async_dispatcher_t` object. A common pattern is to use a single threaded | |
| dispatcher: | |
| - Use an `async::Loop` as the dispatcher implementation. | |
| - Only run one thread to service the loop. | |
| - Only destroy upcall targets on that thread. For example, destroy the | |
| `CsvParser` within a task posted to that dispatcher. | |
| Since the same thread invokes asynchronous callbacks and destroys the instance, | |
| there must be a defined ordering between those operations. | |
| This scenario is common across Fuchsia C++ because FIDL server components are | |
| strongly encouraged to be concurrent and asynchronous. See | |
| [C++ FIDL threading guide][cpp-threading-guide] for a concrete discussion of | |
| this scenario when using FIDL bindings. | |
| ### Sequences | |
| More generally, if a dispatcher promises that tasks posted on that dispatcher | |
| always run with a defined ordering, it is safe to destroy upcall targets on a | |
| dispatcher task, and synchronization with upcalls is guaranteed. Such | |
| dispatchers are said to support *sequences*: sequential execution domains which | |
| runs a series of tasks with strict mutual exclusion, but where the underlying | |
| execution may hop from one thread to another. | |
| Synchronized driver dispatchers ([`fdf::Dispatcher`][fdf-dispatcher] created | |
| with the `FDF_DISPATCHER_OPTION_SYNCHRONIZED` option) are an example of sequence | |
| supporting dispatchers. | |
| When using dispatchers supporting sequences, a common pattern for ensuring | |
| thread safety is to use the object from a single sequence. | |
| ### Enforce with runtime checks {#mutual-exclusion-guarantee} | |
| We provide libraries for enforcing the above common patterns at runtime. In | |
| particular, thread-unsafe types may check that an instance is always used from | |
| an asynchronous dispatcher that ensures ordering between operations. Here "used | |
| from" covers construction, destruction, and calling instance methods. We call | |
| this *mutual exclusion guarantee*. Specifically: | |
| - If the `async_dispatcher_t` supports [*sequences*](#sequences), then code | |
| running on tasks posted to that dispatcher are ordered with one another. | |
| - If the `async_dispatcher_t` does not support sequences, then code running on | |
| tasks posted to that dispatcher are ordered if that dispatcher is only | |
| serviced by a single thread, for example, a single-threaded `async::Loop`. | |
| In short, either the dispatcher supports sequences in which case the object must | |
| be used on that sequence, or the code runs on a single dispatcher thread and the | |
| object must be used on that thread. | |
| The async library offers a BasicLockable type, | |
| [`async::synchronization_checker`](/zircon/system/ulib/async/include/lib/async/cpp/sequence_checker.h). | |
| You may call `.lock()` or lock the checker using a `std::lock_guard` whenever a | |
| function requires mutual exclusion. Doing so checks that the function is called | |
| from a dispatcher with such a guarantee, without actually taking any locks. Here | |
| is a full example: | |
| ```cpp | |
| {% includecode gerrit_repo="fuchsia/fuchsia" gerrit_path="examples/cpp/synchronization_checker/main.cc" region_tag="synchronization_checker" adjust_indentation="auto" exclude_regexp="^TEST|^}" %} | |
| ``` | |
| `fidl::Client` is another example of types that check for mutual exclusion | |
| guarantee at runtime: destroying a `fidl::Client` on a non-dispatcher thread | |
| will lead to a panic. | |
| ### Discard callbacks during destruction | |
| You may have noticed that for the `ChannelReader` example above to work, the | |
| callback passed to `wait_.Begin(...)` must be silently discarded, instead of | |
| called with some error, if `ChannelReader` is destroyed. Indeed the | |
| [documentation][async-wait] on `async::WaitOnce` mentions that it "automatically | |
| cancels the wait when it goes out of scope". | |
| During destruction, some C++ objects would discard the registered callbacks if | |
| those have yet to be called. These kind of APIs are said to guarantee *at most | |
| once delivery*. `async::Wait` and `async::Task` are examples of such objects. | |
| This style works well when the callback references a single receiver that owns | |
| the wait/task, i.e. the callback is an upcall. These APIs are typically also | |
| thread-unsafe and requires the aforementioned mutual exclusion guarantee. | |
| Other objects will always call the the registered callback exactly once, even | |
| during destruction. Those calls would typically provide an error or status | |
| indicating cancellation. They are said to guarantee *exactly once delivery*. | |
| One should consult the corresponding documentation when using an | |
| asynchronous API to understand the cancellation semantics. | |
| It is possible to convert an *exactly once* API into an *at most once* API by | |
| discarding the upcall if the object making the upcalls is already destroyed. | |
| [`closure-queue`][closure-queue] is a library that implements this idea; | |
| destroying a `ClosureQueue` will discard unexecuted callbacks scheduled on that | |
| queue. | |
| ### Use an object with different mutual exclusion requirements | |
| To maintain the mutual exclusion guarantees, one may manage and use a group of | |
| objects on the same sequence (if supported) or single threaded dispatcher. Those | |
| objects can synchronously call into one another without breaking the mutual | |
| exclusion runtime checks. A special case of this is an application that runs | |
| everything on a single `async::Loop` with a single thread, typically called the | |
| main thread. | |
| More complex applications may have multiple sequences or multiple single | |
| threaded dispatchers. When individual objects must be used from their | |
| corresponding sequence or single threaded dispatcher, a question arises: how | |
| does one object call another object if they are associated with different | |
| dispatchers? | |
| A time-tested approach is to have the objects send messages between one another, | |
| as opposed to synchronously calling their instance methods. Concretely, this | |
| could mean that if object `A` needs to do something to object `B`, `A` would | |
| post an asynchronous task to `B`'s dispatcher using `async::PostTask`. The task | |
| (usually a lambda function) may then synchronously use `B` because it is already | |
| running under `B`'s mutual exclusion guarantee. | |
| When tasks are posted to a different dispatcher, it's harder to safely discard | |
| them when the receiver object goes out of scope. Here are some approaches: | |
| - One may shutdown the dispatcher before destroying the object, if that | |
| dispatcher serves exactly that object. For example, `B` may own an | |
| `async::Loop` as the last member field. When `B` destructs, the `async::Loop` | |
| would be destroyed, which silently discards any unexecuted tasks posted to | |
| `B`. | |
| - One may reference count the objects, and pass a weak pointer to the posted | |
| task. The posted task should do nothing if the pointer is expired. | |
| Golang is a popular [example][golang] that baked this principle into their | |
| language design. | |
| ## Prior arts | |
| Lightweight mechanisms of ensuring a set of tasks execute one after the other, | |
| without necessarily starting operating system threads, is a recurring theme: | |
| - The Chromium project defines a similar sequence concept: [Threading and | |
| tasks in Chrome][chrome]. | |
| - The Java Platform added [virtual threads][java]. | |
| [async-readme]: /zircon/system/ulib/async/README.md | |
| [async-loop]: /zircon/system/ulib/async-loop/include/lib/async-loop/loop.h | |
| [async-loop-cpp]: /zircon/system/ulib/async-loop/include/lib/async-loop/cpp/loop.h | |
| [async-wait]: /zircon/system/ulib/async/include/lib/async/cpp/wait.h | |
| [fdf-dispatcher]: /sdk/lib/driver/runtime/include/lib/fdf/cpp/dispatcher.h | |
| [thread-safety]: https://en.wikipedia.org/wiki/Thread_safety | |
| [data-race]: http://eel.is/c++draft/intro.races#21 | |
| [abseil-thread-safety]: https://abseil.io/blog/20180531-regular-types#data-races-and-thread-safety-properties | |
| [cpp-threading-guide]: /docs/development/languages/fidl/tutorials/cpp/topics/threading.md | |
| [closure-queue]: /zircon/system/ulib/closure-queue/include/lib/closure-queue/closure_queue.h | |
| [chrome]: https://chromium.googlesource.com/chromium/src/+/master/docs/threading_and_tasks.md | |
| [java]: https://openjdk.org/jeps/425 | |
| [golang]: https://go.dev/blog/codelab-share |