Pros and cons of Flutter on embedded

What you should know when getting started with Flutter on Embedded Systems

Flutter is a cross-platform UI framework mainly developed by Google, that up until now has mostly been used for mobile and desktop apps. In contrast to other solutions, the apps are written in a unique programming language called Dart and run fast and natively on the end device. In this blog post, I will report on my experiences with Flutter on embedded systems.

Flutter is Google's attempt to solve the problem of “cross-platform apps” once and for all. It uses a unique programming language called Dart, which has actually been in existence for quite some time now but was mostly a “solution looking for a problem” before Flutter. Dart supports multiple runtime modes (Debug, Profile, Release) and executes code in JIT- or AOT-mode, depending on the mode. As opposed to other frameworks, Flutter does not make use of the platform's native UI-components, for example UIKit, but constructs its own OpenGL / Metal / Vulkan graphics surface and does completely custom rendering using Skia. This, in combination with AOT-mode, not only makes for a good performance advantage but also ensures the app looks and behaves as expected on every platform; although this could also be disadvantageous sometimes. So, as Flutter has pretty low requirements on the underlying native platform, it’s not that difficult to make it work in embedded environments. Only an OpenGL- or Vulkan interface is required.

Flutter on Raspberry Pi

I found that interesting, so I decided in 2019 to make Flutter work on a Raspberry Pi. My intention was to build my own custom car-infotainment system, with digital radio, Android Auto support, CAN-bus and Google-Assistant connection. I got the idea because my very simple car back then didn’t have enough space for a normal, aftermarket touchscreen stereo.

I put this project on GitHub under the name “flutter-pi”, where it has a sizable user base at this point. However, nowadays I mostly work on flutter-pi or related parts instead of the actual infotainment system.

Flutter Engine Embedder

Flutter-pi is a so-called Flutter Engine Embedder.

As you can see in the figure, Flutter is roughly divided into 3 components. First there is the part that is written in Dart, the Flutter Framework. In the framework there are e.g. the "Widget Tree", "Element Tree" and "Render Object Tree", which describe the user interface and probably look familiar if you have ever written a Flutter app. Nothing is rendered in the framework yet, but the rendering intent is recorded there and passed to the Flutter engine. The engine then does the heavy lifting and converts it all (using Skia) into draw calls for the graphics library and passes the rasterized image to the embedder, which is tasked with presenting the image on the screen.

The Flutter Engine provides an API called the "Embedder API". This API is used by so-called embedders to make Flutter on a platform. Each supported platform has its own embedder. For example, there are official embedders for Android and iOS, written in Java and Objective-C respectively, and official embedders for the desktop platforms & Fuchsia. There are also external (3rd party) embedders from various developers: ivi-homescreen by Toyota, flutter-elinux by Sony and flutter-pi, by me, all developed for Linux-Embedded.

Framework, engine and embedder of course have other tasks besides rendering. The embedder provides the engine with input events (touchscreen, mouse, keyboard). In addition, the embedder can provide an API for native plugins, which can then be used by a video player plugin, for example. The engine does not provide such an API for plugins, which means that this plugin API may look different for each embedder. So the "write-once, run-anywhere" concept falls away, at least for native plugins. In addition, the engine provides the embedder with a "semantics tree" that maps the interactive elements of the user interface as semantics nodes so that the embedder can implement accessibility features.

Flutter-Pi

The unique thing about flutter-pi is that it uses the KMS/DRM kernel APIs directly. There is no detour via windowing systems like Wayland or X11. This has advantages, but also disadvantages. Advantageous is, for example, that the overhead is lower and you can integrate Flutter very deeply with the platform (or the KMS API on the platform). A downside is, however, that you partly have to re-solve all the problems and challenges that the Linux Kernel Modesetting API poses to the user and were already solved in Wayland or X11.

Pros and Cons of Flutter

A major unique selling point of Flutter is the superior development experience. The SDK can be installed in just a few steps. After installation, the SDK is mostly used via the so-called "flutter tool" or the IDE (Visual Studio Code or IntelliJ). The tool combines all the functionality of the SDK in a single command line program. Creating an app is a simple "flutter create" and launching the app "flutter run". If anything doesn't work, "flutter doctor" provides diagnostic information and troubleshooting tips.

Probably the most interesting development feature of Flutter is the state-preserving "hot-reload". If you change the color of a button in the source code, you will see the change in the running app within about half a second. By the way, this works not only with colors, but also with complex logic, if necessary. The rest of the app's state is preserved.

In other places, too, you can see that the Flutter team puts a lot of emphasis on the development experience. The IDE integration is very good. Most typical refactorings are supported directly in the IDE and for many common changes in the source code there are templates that reduce the writing work. Virtually for every Dart Analyzer warning or hint there are direct IDE suggestions to fix it.

The fact that Flutter originally comes from the mobile space also has advantages for embedded. New developments in the mobile space are immediately available for embedded, e.g. the new Material 3 design (that's not completely implemented yet though). In addition, Flutter uses the BSD 3-Clause license, which is similar to the MIT license and thus very permissive.

Flutter is popular and the trend is upward: currently about 500,000 users use the SDK monthly. With this number of users, it naturally has an active ecosystem, with many externally developed packages. However, the fact that Flutter is mainly developed by Google is a concern for many embedded developers. Google projects come to a very sudden end in some cases, and embedded products are often designed for long cycles. Of course, the Flutter team is doing its best to put these concerns into perspective. Personally, I think a premature end is actually rather unlikely, as Google is already far too deeply involved at this point.

Furthermore, the embedded area still has some special requirements that have not yet been addressed in Flutter, and in fact Google is not very interested in Flutter working well on Linux embedded. All Flutter engine embedders that are suitable for Linux embedded are developed externally. The omnivalent flutter tool also works only very limitedly with embedded devices (via a feature called "custom-devices", which I developed), though at least the main features like hot-reload, hot-restart and the Flutter devtool work. Plugins that contain native C/C++ code, for example, do not work with the tool yet. (These currently have to be written directly into the embedder) It is worth mentioning, however, that Sony is developing its own Flutter tool that can do just that.

Overall, it can also be said that integration with native code in Flutter is generally rather laborious. The "official" way to let native code communicate with Dart code is via message passing (so-called platform channels). This message passing requires some boilerplate code. A remedy can be codegen packages, or instead, communication via foreign function interface. Nevertheless, since the plugin API is provided by the embedder, as mentioned above, and not by the engine, native code must sometimes be adapted individually for each embedder.