While an engineer is still in the same headspace of writing a feature, if they see a bug in the code that they've written, they're able to immediately fix that before it even gets to our manual QA team.
The Challenge
Manual testing creates a heavy workload for a lean workforce
Since users rely on AllTrails for navigation of over 400,000 trails in a variety of conditions, it was critical to get as close to bug-free as possible. Unfortunately, James Graham, CTO, and Adam Blumberg, QA Manager, found that updates were taking weeks to build and test.
“Before every release, we would rely on other cross functional teams to run a regression suite on the app,” said James Graham, CTO of AllTrails. ”This could take anywhere from three to four days.”
The company was growing quickly and needed to ship more often, at a regular cadence. Taking cross functional teams away from their primary work to manually test was not aligned with this greater business need.
“We wanted to ship much more quickly than we’d been shipping in the past,” said Adam Blumberg, QA Manager at AllTrails. “We didn’t want to wait two, three, four weeks to get things out the door.”
The Solution
Help the team ship quickly while protecting critical flows
Waldo for AllTrails QA
Graham and Blumberg set out to find automated end-to-end testing for their mobile app. They knew they didn’t want the technical debt that came with engineers having to work on scripts, so they chose to partner with Waldo.
Waldo removed the burden of having to write and maintain scripts from the engineering team—the QA team can now create tests themselves: “Waldo’s interface makes it really easy for our QA team to edit, test, and modify each test,” Graham said. “It's a more seamless experience than managing and maintaining scripts.”
“Waldo is great for allowing people who aren't software engineers to jump right in and build a test quickly,” Blumberg added.
Instead of pulling people away from customer support for manual testing, AllTrail’s existing QA team can create and maintain a fully automated, comprehensive suite that runs nightly.
Waldo for AllTrails Engineers
Simultaneously, engineers wanted to ensure that their critical user flows were not broken as updates were being made in real time. These critical flows include:
- Login
- Signup
- Navigating to a trail page
- Looking at a map
Taking advantage of the Branches feature in Waldo, engineers can have a small subset of smoke tests running before their feature branch PRs are merged. This ensures total coverage on the most important parts of their app with every single change.
“That means the engineers are able to work on the issues before they reach the manual QA team—as opposed to being notified later, when the engineer has moved on to something else,” Blumberg said. “Once you've moved on, it's hard to put yourself back in the headspace of that code, and the bug doesn't immediately get fixed.”
The Result
With Waldo’s seal of approval, issues don’t fall through the cracks
With Waldo, AllTrails can have confidence that what they ship will never break a critical flow. Testing that previously took 3-4 weeks to complete manually, now can be run multiple times throughout their development lifecycle. About 60% of AllTrails’s QA on the app is now covered by Waldo.
“We know that Waldo will run a regression suite every day and find any bugs or gaps in the UI that may change from the previous version of the app,” Graham said. “The customer experience is what we're validating.”
By taking time off the manual QA team’s plate and allowing engineers to respond to issues in real time, AllTrails can maintain their desired velocity without unnecessary risk.
“It gives everybody high confidence when we have that Waldo seal of approval,” Blumberg said.
The FUTURE
We're able to ship a lot quicker in our release cycle. The amount of coverage that we have on the mobile app, it allows us to take those cases off of our manual team's plate.
ABOUT THE CUSTOMER
AllTrails is home to the largest collection of digital trail guides in the world. The company was founded on the idea that we’re all made better by spending time in nature. Today, the company continues to be driven by the desire to share the outdoors with as many people in as many places as possible—and to do so responsibly and respectfully.