If you’ve read the introductory documentation and played a bit with Waldo Scripting, you may be wondering how to run scripted tests from your CI on an existing mobile app. I wondered too, and so I took the time to figure out how to do exactly that.
I took a stripped-down version of TravelSpot, a demo iOS app that the Waldo team has used in years past to illustrate the power of Waldo for automated testing, and successfully added support for end-to-end testing with Waldo Scripting by using a GitHub Actions workflow.
Now, whenever I create a PR that targets the main
branch, my end-to-end tests run automatically, plus I am not allowed to merge in my changes until all the tests pass.
I spent several hours figuring this all out, but you can hitch a ride on my coattails and spend just a few minutes following along as I describe step by step how I did it.
Preliminaries
First off, I took care of some preliminary tasks:
- I visited here and signed up for a Waldo account.
- I then duly worked my way through the “Getting Started” guide so I could familiarize myself with what Waldo Scripting is actually capable of.
At this point, with my app uploaded, I was ready to roll up my sleeves and get to the real work!
Integrating Waldo Scripting
I decided that my first order of business would be to add all the JavaScript glue required to run Waldo Scripting into my app repo.
Cloning the Waldo Scripting samples repo
After a bit of experimentation, I determined that the easiest approach was to clone the Waldo Scripting samples repo directly into a subfolder of the top-level app folder:
To keep things simple, I called the new subfolder waldo
right from the get go.
Deleting extraneous files and folders
Since I didn’t want to end up with another repo nested within my app repo, I immediately deleted the .git
folder inside the waldo
subfolder:
Now that I was in the waldo
subfolder, I went ahead and also nuked the samples
folder since it was highly unlikely that I would need anything from there:
I also knew that the tests
subfolder was where all of my (future) tests would live, so I scrapped the existing scaffold-test.ts
file. Given that I already had an initial E2E test script in mind, I went ahead and created a new (empty) test script file called onboarding.
Writing the Test Script
It seemed to me that writing the onboarding.ts
test script first made the most sense. That way I’d be able to test it out immediately by simply invoking the same command I had previously used to complete the third step of the “Getting Started” guide:
It took a few iterations before I had a test script that consistently passed. Here’s what I ultimately ended up with:
And here is the public replay link of a successful run of that test script.
Now I just needed to be able to run that test script from my CI.
Writing the GitHub Actions Workflow
I decided to make things easy and implement a GitHub Actions workflow that
- builds the app,
- uploads it to Waldo, and
- runs the end-to-end tests against it.
While it sounds easy, there were a few wrinkles to be ironed out. This is what I cobbled together in the end:
I will point out the places that deserve special attention.
Locating the build artifact
Building the TravelSpot
app with xcodebuild
was reasonably straightforward; however, building it such that I could easily locate the resulting build artifact (meaning TravelSpot.app
) — and subsequently upload it to Waldo — required specifying an explicit derived data path:
That last line is a bit of GitHub Actions magic to save the location of the build artifact in an environment variable named APP_DIR_PATH
so that it can be used later on in the workflow.
I’ve got a secret!
Uploading the TravelSpot.app
build artifact was super simple with Waldo’s custom upload action. I could easily supply the artifact’s location from the APP_DIR_PATH
environment variable set in the preceding step:
Uploading to Waldo requires you to specify the “CI token” associated with the app you are targeting. It can be found in the “General” tab of the “Configuration” for that app on the Waldo dashboard:
For obvious security purposes, I stashed this little jewel as a repository secret named WALDO_CI_TOKEN
in GitHub:
GitHub Actions, of course, made it dead simple to access this little secret from my workflow.
Notice that, unlike any of the other steps in this workflow, I specified an id
on this particular step. (I cleverly called it upload
.) The reason for this will become apparent momentarily.
Running the test scripts
After a couple more steps to get Node.js properly configured, it was time to actually run the test scripts. Thus I used our familiar friend npm run wdio
, but with a couple of twists:
Notice how I was able to use the build ID from Step 3 as the version ID in this step. Starting with v2.0.0, Waldo’s custom upload action provides a build_id
output upon successful upload to Waldo. Specifying the id
as upload
on Step 3 allowed me to access that build ID with ${{ steps.upload.outputs.build_id }}
on Step 6.
As you no doubt saw in the “Getting Started” guide, running a test script against a remote (Waldo-provided) device requires authentication. The guide uses the command invocation npm run authenticate [token]
for this purpose. I decided to use a handy shortcut instead: defining the TOKEN
environment variable allows the npm run wdio
command to authenticate on the fly. Of course I once again made use of the WALDO_CI_TOKEN
repository secret.
It’s a Wrap!
And that, my friends, is how I was able to successfully add support for end-to-end testing with Waldo Scripting using a GitHub Actions workflow.
Ah! The sweet smell of success!
Automated E2E tests for your mobile app
Get true E2E testing in minutes, not months.