Maestro: A modern, flexible, and reliable test automation solution that really does "just work"

In the ever-accelerating world of mobile app development, maintaining quality while rapidly iterating can feel like an impossible balancing act. Manual testing is time-consuming and prone to human error, while traditional automation frameworks are often complex, brittle, and require specialized skills. Enter Maestro, the open-source mobile UI automation framework that "just works".

Introduction

Just over a year ago, I found myself looking for a new way to automate, here were the requirements:

• Low barrier to entry: I needed a tool that was easy to pick up and didn't require any programming experience. I wanted it to be accessible to anyone in the business, from designers and business analysts to manual testers with no automation experience and also developers.
• Capable: I needed a tool that could automate UI tests for Android and iOS across multiple devices and OS versions while supporting WebViews
• Reliable: I wanted a tool that didn't break every time there was a full moon
• Flexible: I wanted the flexibility to run the tooling on our infrastructure so we could maximize the amount of automated testing we could execute without worrying about the costs

I've worked with Appium and similar solutions but was unimpressed by their brittle nature, high barrier to entry, and maintenance overhead. I also looked at enterprise solutions and winced at their eye-watering costs.

When I came across Maestro, a relatively new solution, I was excited, it appeared to do everything I needed and claimed to "just work". After a year, here's what I've learned about Maestro.

What Makes Maestro a Maestro?

Maestro is designed from the ground up to simplify and streamline mobile UI testing. And best of all, it really does "just work", at least more times than it doesn't 😉.

1. Declarative Syntax (YAML): Maestro uses a simple and human-readable YAML format to define test flows. You don't need to be a programming guru to write tests. The intuitive structure makes it easy to describe actions like tapping buttons, entering text, and verifying elements on the screen.
2. Cross-Platform Compatibility: Maestro supports iOS, Android, React Native, Flutter, and web views.
3. Built-in Resilience: Mobile apps can be flaky. Network issues, animations, and unexpected UI changes can all cause traditional test scripts to fail. Maestro handles these challenges gracefully with smart waiting mechanisms and retry logic, making your tests more robust and reliable.
4. Blazing Fast Iteration: Maestro doesn't require compilation, which means you can see the results of your changes instantly. This rapid feedback loop speeds up development and helps you catch issues early.
5. Powerful Extensibility: While Maestro provides a wide range of built-in actions, you can easily extend it with custom actions written in JavaScript. This allows you to tailor Maestro to your specific testing needs. Due to the nature of the apps we're testing day-to-day many app functions rely on something to have happened externally. By using simple scripts in Maestro I can reach out to external cloud-hosted systems to initiate those external processes to simulate real-world usage, which is very useful indeed.
6. Maestro Studio (GUI): For those who prefer a visual approach, Maestro Studio provides a graphical interface for creating and managing test flows. It makes it even easier to get started with Maestro, especially for those new to automation. This is typically how I start most tests. Fire up an emulator and then use Maestro Studio to step through the tests I want to write. It'll effortlessly create the YAML you'll need for your test files. Furthermore, this makes debugging easy, you can see all the IDs Maestro can "see" so you know the best way to target specific elements.
7. Community: This is a big one. The community is amazing! Maestro has a vibrant Slack channel full of awesome people happy to help and provide advice. I've relied on this community several times to get me out of a fix and I'm very grateful. When choosing to adopt a new technology ensuring there is a strong community behind it is essential, for Maestro it's a 10/10.
8. Maestro Cloud & CI/CD Integration: Maestro Cloud takes the convenience and power of Maestro to the next level. By leveraging the cloud, it eliminates the need for maintaining physical devices or complex testing infrastructure. With Maestro Cloud, you can seamlessly run your automated tests on emulators hosted in the cloud. This then integrates effortlessly with popular CI/CD platforms, allowing you to run your tests automatically as part of your build and deployment process.

Where things aren't quite so perfect

During the time I've been using Maestro, it's proven to be a reliable tool, but there are a small number of things to be aware of before diving in.

Studio and CLI Conflicts

For whatever reason it doesn't appear to be possible to run Maestro Studio at the same time tests are running. When I do this the driver times out intermittently and the tests fail. This means that when you're debugging your tests, you must remember to kill Maestro Studio before running your tests, and then re-launching Maestro Studio when it's time to do further inspection. I suspect this is some limitation in how the simulator drivers function, I'm not sure. If you do know, please let me know in the comments!

Cloud Limitations

Maestro has done a great job with their cloud offering, like the rest of their tooling, it "just works". The issue I've come across is that it's so popular you're often waiting a long time before the simulator type you want to test on is available. If you're testing on older OS versions then you might be OK, but I'm assuming most people will want to test on at least the latest OS version. As a result, I've implemented my own system that runs my tests nightly, grabs device logs, and screenshots, records a video of the test run, and then outputs everything to a nice HTML webpage for review.

You are also limited to a 7-minute execution time for each test with Maestro Cloud. For most, this won't be a problem and it often makes sense to keep tests small but seems overly restrictive for some use cases.

No Real Device Support on iOS

When I started this journey I envisioned something like the above, however, Maestro doesn't currently support testing on physical iOS devices. This might not be a problem depending on your app and use case given how capable the iOS simulator is but something you should consider before adopting Maestro. The good news is that you can test on physical Android devices (Android is better than iOS anyway 😉).

And that's it. There are some smaller quirks and known issues which you can read here, like how the iOS hideKeyboard command often doesn't work properly, but there is no native command for Maestro to hook into, which is hardly Maestro's fault.

Getting Started with Maestro

Convinced it's worth a go? If you're ready to experience the magic of Maestro, here's how to get started. You can read the full guide in the documentation.

Install Maestro

Run this simple command to download and install the latest version of Maestro.

curl -fsSL "https://get.maestro.mobile.dev" | bash

Note: If you're on MacOS you need to make sure you have Command Line Tools installed (Xcode -> Preferences -> Locations -> Command Line Tools).

Create a virtual device

To run your tests, you'll need to create a virtual device.

To create an Android emulator:

avdmanager create avd --force --name maestro-sim-1 --device "pixel_6a" --abi arm64-v8a --package "system-images;android-34;google_apis_playstore;arm64-v8a"

To create an iOS simulator:

xcrun simctl create maestro-sim-1 com.apple.CoreSimulator.SimDeviceType.iPhone-11 com.apple.CoreSimulator.SimRuntime.iOS-17-5

Install your app

Next, you'll need to ensure that the app you want to test has been installed on the iOS simulator or Android emulator.

To install your app on an Android emulator via the CLI:

1. Boot the emulator you've just created:

emulator -avd maestro-sim-1 -no-snapshot -no-boot-anim -wipe-data

2. Once booted, install your app:

adb install /Path/to/Your.apk

To install your app on an iOS simulator via the CLI:

1. Boot the simulator you've just created:

xcrun simctl boot maestro-sim-1

2. Install your app:

xcrun simctl install booted /Path/to/Your.app

Write your first test

1. Create a .yaml file called first-test.yaml with the following content, using your app ID and modifying text to tap to match something tappable in your app:

# first-test.yaml

appId: your.app.id
---
- launchApp
- tapOn: "text to tap"

Run your first test

With only one simulator/emulator running, execute the following command to run your first test!

maestro test first-test.yaml

You should then get some output that looks something like this:

If you run into any issues please review the documentation.

Conclusion

In conclusion, Maestro stands as a testament to the power of simplicity and efficiency in the realm of mobile UI automation. Its intuitive YAML syntax, cross-platform compatibility, built-in resilience, and rapid iteration capabilities significantly reduce the barriers to entry and maintenance overhead associated with traditional automation frameworks. While a few limitations and quirks persist, the active community and ongoing development suggest a bright future for Maestro. Whether you're a seasoned automation engineer or a newcomer to the field, Maestro offers a compelling solution to streamline your mobile testing efforts and ensure the delivery of high-quality mobile applications.