This article is a must-read for those experiencing headaches with mobile automation testing in the context of Continuous Integration and Continuous Deployment (CICD), particularly when it involves native mobile Apps for Android and iOS. It’s quite challenging to find sufficient resources that cover this specific topic.

Within this article, I will guide you through a detailed step-by-step process on how to create a comprehensive end-to-end test pipeline at no cost, utilizing GitHub Actions for both the iOS and Android platforms. We will be using our beloved WebdriverIO framework throughout the tutorial.

Challenge​

Our challenge is to establish a unified pipeline workflow that enables testing of our native mobile application on both the iOS and Android platforms. In a previous article, we thoroughly explored the process of constructing a pipeline for testing an Android app using an emulator with GitHub Actions. To handle the E2E test for the Android component, we will reuse that workflow. However, we still need to address the remaining bigger challenge of creating a separate job for the iPhone/iPad simulator.

During my research, I came across an incredibly useful but often overlooked feature which is, the macOS GitHub runner comes pre-equipped with Xcode, the necessary SDKs for iPhone iOS simulators. This realization sparked an idea in my mind: why not replicate the process we followed for the Android emulator but this time for the iOS simulator? This is particularly exciting because GitHub Actions’ iOS runner supports virtualization, making it a viable option for our purposes.

This feature allows us to build our pipeline without any additional costs (up to a maximum of 2000 minutes).

Workflow structure​

Our GitHub actions workflow will be basically the same as Android in the core concept but with a little technical variation

• Create simulator
• Install dependencies
• Execute the test
• Generate the report

Looks pretty straightforward, but how? Let’s see

Step 1​

As previously mentioned, the GH Actions runner comes packed with a range of available simulators. While we could utilize one of these existing simulators, it would require using the deviceName and it's

randomly changing UUID for each execution. However, you can still extract the relevant UUID using shell commands.

To simplify the process and increase flexibility, we will create our own simulator. Since Xcode is already installed, we can make use of the “xcrun” CLI. To create a simulator from the installed iOS versions using the terminal, simply execute the following command:

xcrun simctl create "iPhone 14 Pro" "com.apple.CoreSimulator.SimDeviceType.iPhone-14-Pro" "com.apple.CoreSimulator.SimRuntime.iOS-16-0"

Executing this command will result in the immediate creation of a simulator and the subsequent retrieval of its UUID.

To enhance re-usability and optimize the process, we can encapsulate this command within a shell script. With a few modifications, we can ensure that the UUID is stored as an environment variable in GitHub Runner which we will eventually use for our test capabilities.

#!/bin/bash

# Set iPhone model and iOS version
iphone_model="${IPHONE_MODEL// /-}"
ios_version="${IOS_VERSION//./-}"
simulator_name="${iphone_model}"
simulator_udid=$(xcrun simctl create "$IPHONE_MODEL" "com.apple.CoreSimulator.SimDeviceType.$iphone_model" "com.apple.CoreSimulator.SimRuntime.iOS-$ios_version")

# Export the simulator UDID as an environment variable
export SIMULATOR_UDID="$simulator_udid"
echo "SIMULATOR_UDID=$SIMULATOR_UDID" >> $GITHUB_ENV

# Boot the simulator
xcrun simctl boot "$simulator_udid"

By using the script above, we can provide the device model and iOS version as environment variables which can be stored in the environment sections in our workflow, this will create the simulator and store its UUID in GITHUB_ENV. This UUID will be essential for configuring the desired capabilities in our tests.

Since we are using IPHONE_MODEL and IOS_VERSION as environment variable in our shell script then we will have to set them in the environment section as shown below.

Step 2​

After successfully creating and booting up the simulator in the previous step, it’s crucial to verify that the process was completed without any issues and that the device is fully prepared for use.

To ensure the successful starting of our test, it is crucial to confirm that the IOS has fully booted. For this purpose, I have created a code snippet that continuously monitors the device’s status until a specific output is obtained, signifying the completion of the simulator’s booting process.

#!/bin/zsh

function wait_for_boot() {
  printf "${G}==> ${BL}Waiting for the simulator to boot...${NC}\n"
  start_time=$(date +%s)
  spinner=( "⠹" "⠺" "⠼" "⠶" "⠦" "⠧" "⠇" "⠏" )
  i=0
  # Get the timeout value from the environment variable or use the default value of 60 seconds
  timeout=${BOOT_TIMEOUT:-60}

  while true; do
    output=$(xcrun simctl bootstatus "$SIMULATOR_UDID")
    echo "${output}"
    if [[ $output == *"Device already booted, nothing to do."* ]]; then
      printf "\e[K${G}==> \u2713 Simulator booted successfully${NC}\n"
      exit 0
    else
      printf "${YE}==> Please wait ${spinner[$i]} ${NC}\r"
      i=$(( (i+1) % 8 ))
    fi

    elapsed_time=$(( $(date +%s) - $start_time ))
    if [[ $elapsed_time -ge $timeout ]]; then
      printf "${RED}==> Timeout waiting for simulator to boot 🕛${NC}\n"
      exit 1
    fi

    sleep 1
  done
}

# Call the wait_for_boot function
wait_for_boot

Step 3​

Proceeding further, we will cover the necessary steps and dependencies required for executing your tests. This includes the installation of Appium, the XCUITest driver, and the essential Node.js libraries.

  "devDependencies": {
    "@wdio/allure-reporter": "^8.10.4",
    "@wdio/appium-service": "^8.10.5",
    "@wdio/cli": "^8.10.5",
    "@wdio/local-runner": "^8.10.5",
    "@wdio/mocha-framework": "8.10.4",
    "@wdio/spec-reporter": "8.8.7",
    "ts-node": "^10.9.1",
    "typescript": "^5.0.4"
  },
  "dependencies": {
    "allure-commandline": "^2.22.1",
    "appium": "2.0.0-beta.71",
    "appium-uiautomator2-driver": "*",
    "appium-xcuitest-driver": "*"
  }

Connecting the Puzzle Pieces​

Since now the key elements required to set up the environment for executing our mobile automation tests on the iOS simulator are ready, Let’s wrap them all into a single yaml file for GH actions

name: Wdio-x-native

on:
  workflow_dispatch:

env:
  IPHONE_MODEL: iPhone 8
  IOS_VERSION: 16.2
  BOOT_TIMEOUT: 700

jobs:
  ios:
    runs-on: macos-13

    steps:
      - uses: actions/checkout@v3

      - name: Export environment variables
        run: |
          export IPHONE_MODEL=$IPHONE_MODEL
          export IOS_VERSION=$IOS_VERSION

      - name: Start simulator
        run: |
          chmod a+x ./sscript/start_simu.sh
          ./sscript/start_simu.sh

      - name: Install dependencies
        run: |
          npm i

      - name: Check simulator booting status
        run: |
          chmod a+x ./check_simu.sh
          ./check_simu.sh

      - name: Execute the test
        run: |
          npm run ios

Combine the simulator status check and simulator start into a single shell script would have been possible. However, I intentionally separated them to execute them individually. This allows me to utilize the time taken for the simulator to boot up and install the remaining dependencies. Afterwards, I can then proceed to check the status of the simulator. Similarly we will apply same approach to Android emulator (check previous article).

Build cross-platform workflow​

The time now to combine our Android workflow from the previous article without Ios workflow into one single workflow, using the matrix strategy as follows:

name: Wdio-x-native

on:
  workflow_dispatch:

env:
  IPHONE_MODEL: iPhone 8
  IOS_VERSION: 16.2
  API_LEVEL: 32
  EMULATOR_NAME: Nexus
  EMULATOR_DEVICE: Nexus 5
  EMULATOR_VERSION: 12
  ANDROID_ARCH: x86_64
  ANDROID_TARGET: google_apis
  ANDROID_BUILD_TOOLS_VERSION: 34.0.0-rc4
  ANDROID_SDK_PACKAGES: system-images;android-32;google_apis;x86_64 platforms;android-32 build-tools;34.0.0-rc4 platform-tools emulator
  EMULATOR_TIMEOUT: 350
  BOOT_TIMEOUT: 700

jobs:
  ios:
    runs-on:
       - macos-13
    strategy:
      matrix:
        os: [IOS]
        device: [$IPHONE_MODEL]
        version: [$IOS_VERSION]
    steps:
      - uses: actions/checkout@v3

      - name: Export environment variables
        run: |
          export IPHONE_MODEL=$IPHONE_MODEL
          export IOS_VERSION=$IOS_VERSION
    # ...
    # find the full workflow at the end of the article
    # ...

  android:
    runs-on: macos-13
    strategy:
      matrix:
        os: [Android]
        emulator_name: [$EMULATOR_NAME]
    steps:
      - uses: actions/checkout@v3

      - name: Add avdmanager and sdkmanager to system PATH
        run: |
          echo "$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/emulator:$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$ANDROID_HOME/build-tools/${{ env.ANDROID_BUILD_TOOLS_VERSION }}" >> $GITHUB_PATH

      - name: Install Sdk
        run: |
          yes Y | sdkmanager --licenses
          sdkmanager --install ${ANDROID_SDK_PACKAGES}

      - name: Build emulator

    # ...
    # find the full workflow at the end of the article
    # ...

In the above example, we have integrated the IOS workflow mentioned earlier with the Android emulator workflow described in our previous article.

These are the recommended configurations that you may require for both the Android emulator and iPhone simulator. It’s important to note that the deviceName, platformVersion, and UUID are not hardcoded in our object. This flexibility allows us to easily switch between different versions and device models as needed.

const emulator = [{
  platformName: 'android',
  'appium:options': {
    deviceName: process.env.CI ? process.env.EMULATOR_NAME : 'Nexus',
    platformVersion: process.env.CI ? process.env.EMULATOR_VERSION : '13',
    automationName: 'uiautomator2',
    appPackage: 'com.wdiodemoapp',
    appWaitPackage: 'com.wdiodemoapp',
    appActivity: 'com.wdiodemoapp.MainActivity',
    appWaitActivity: 'com.wdiodemoapp.MainActivity',
    uiautomator2ServerLaunchTimeout: 200000,
    uiautomator2ServerInstallTimeout: 200000,
    appWaitForLaunch: true,
    autoGrantPermissions: true,
    adbExecTimeout: 200000,
    androidInstallTimeout: 150000,
    ignoreHiddenApiPolicyError: true,
    noReset: true,
    fullReset: false
  }
}]

const simulator = [{
  platformName: 'iOS',
  'appium:options': {
    deviceName: process.env.CI ? process.env.IPHONE_MODEL : 'Iphone-13',
    platformVersion: process.env.CI ? process.env.IOS_VERSION : '15.5',
    automationName: 'XCUITest',
    bundleId: 'org.wdioNativeDemoApp',
    app: 'iOS-Simulator-NativeDemoApp-0.4.0.app.zip',
    udid: process.env.CI ? process.env.SIMULATOR_UDID : '15A098DB-B8A0-4D6A-9057-23FF1F0F0D9B',
    useNewWDA: true,
    usePrebuiltWDA: false,
    wdaConnectionTimeout: 180000,
    appWaitForLaunch: true,
    noReset: true,
    fullReset: false
  }
}]

Initial Execution​

The good news is that workflow is configured properly and the e2e test for the IOS app has been successfully executed

Though the end-to-end test for the iPhone simulator has passed, it was observed that the test for the Android emulator shows instability.

Debugging​

System UI Crush​

It appears that running Android for the first time in headless mode occasionally results in random system UI unresponsive issues. Unfortunately, this issue is preventing us from executing the tests as the UI system is unresponsive which consequently disrupts Appium from proper interaction with the app.

The issue was confirmed when reviewing the allure report screenshots

This explains why the terminal log displayed that Appium was unable to locate any element, despite the successful launch of the app.

This makes sense as Appium is trying to find the desired element but on the current running activity which is .systemui, even though our target app is launched in the background

Connection timeout​

It has been noted that on certain occasions, Appium encountered failures in initiating the test, with all connection retry attempts proving unsuccessful. However, after conducting a thorough investigation, it was discovered that the installation of the Apk file to the Android emulator through the app:“./test.apk” capability was taking an unusually long time, requiring a significantly extended connection timeout to ensure successful installation which is not the best solution.

Now that we have identified the issue and its root cause, it’s time to address and resolve them.

Resolving​

System UI Crush​

Fortunately, we can utilize the advantage of being able to grep the current running activity on an Android device. This privilege allows us to detect whether the system UI or any similar Android service will crash or function normally. We can achieve this by executing the following adb shell command:

adb shell dumpsys window 2>/dev/null | grep -i mCurrentFocus

In our ongoing implementation, we can mimic our natural behavior when encountering this issue on an Android device. Specifically, we will continuously click the home button until the issue is resolved. Once the problem is resolved and the Android system is functioning correctly, we anticipate observing the “.NexusLauncherActivity” as the current main activity running (where “Nexus” represents the Android device).

In order to achieve this, I have developed the following shell script:

#!/bin/bash

function check_current_focus() {
  printf "==> Checking emulator running activity \n"
  start_time=$(date +%s)
  i=0
  timeout=20
  target="com.google.android.apps.nexuslauncher.NexusLauncherActivity"

  while true; do
    result=$(adb shell dumpsys window 2>/dev/null | grep -i mCurrentFocus)

    if [[ $result == *"$target"* ]]; then
      printf "==>  Activity is okay: \n"
      printf "$result\n"
      break
    else
      adb shell input keyevent KEYCODE_HOME
      printf "==> Menu button is pressed \n"
      i=$(( (i+1) % 8 ))
    fi

    current_time=$(date +%s)
    elapsed_time=$((current_time - start_time))
    if [ $elapsed_time -gt $timeout ]; then
      printf "==> Timeout after ${timeout} seconds elapsed 🕛.. \n"
      return 1
    fi
    sleep 4
  done
}

check_current_focus

The shown function above will continuously loop, If the main activity is not found (NexusLauncherActivity), it will send a home button event and repeat the process until its found or timeout is reached.

Connection timeout​

Rather than significantly extending the Appium connection timeout, I will handle the APK installation in a separate step along with the main activity checking.

      - name: Install APK
        run: |
          adb install Android-NativeDemoApp-0.4.0.apk
          chmod a+x ./mainActivityCheck.sh
          ./mainActivityCheck.sh

Excellent! Our solution has been executed successfully with the proper installation of the APK. As expected, the system UI was not responsive, and the shell script effectively managed and handled the situation.

Optimizing and Enhancing workflow​

I have improved the workflow dispatch to provide better control over the platforms on which I can execute my tests, whether it be iOS, Android, or cross-platform.

name: Wdio-x-native

on:
  workflow_dispatch:
    inputs:
      e2e:
         type: choice
         description: Select a platform
         required: true
         options:
            - xplatform
            - ios
            - android
         default: xplatform

Consequently, our jobs should be adjusted

name: Wdio-x-native

on:
  workflow_dispatch:
    inputs:
      e2e:
         type: choice
         description: Select a platform
         required: true
         options:
            - xplatform
            - ios
            - android
         default: xplatform


permissions:
  contents: write
  pages: write
  id-token: write

env:
  IPHONE_MODEL: iPhone 8
  IOS_VERSION: 16.2
  API_LEVEL: 32
  EMULATOR_NAME: Nexus
  EMULATOR_DEVICE: Nexus 5
  EMULATOR_VERSION: 12
  ANDROID_ARCH: x86_64
  ANDROID_TARGET: google_apis
  ANDROID_BUILD_TOOLS_VERSION: 34.0.0-rc4
  ANDROID_SDK_PACKAGES: system-images;android-32;google_apis;x86_64 platforms;android-32 build-tools;34.0.0-rc4 platform-tools emulator
  EMULATOR_TIMEOUT: 350
  BOOT_TIMEOUT: 700

jobs:
  ios:
    runs-on:
       - macos-13
    if: ${{ contains(github.event.inputs.e2e, 'ios') || contains(github.event.inputs.e2e, 'xplatform') }}
    strategy:
      matrix:
        os: [IOS]
        device: [$IPHONE_MODEL]
        version: [$IOS_VERSION]
    steps:
      - uses: actions/checkout@v3
    # find the full workflow at the end of the article

  android:
    runs-on: macos-13
    if: ${{ contains(github.event.inputs.e2e, 'android') || contains(github.event.inputs.e2e, 'xplatform') }}
    strategy:
      matrix:
        os: [Android]
        emulator_name: [$EMULATOR_NAME]
    steps:
      - uses: actions/checkout@v3
    # find the full workflow at the end of the article

Finally, Generate our report and deploy it to the GitHub page out of the box

      - name: Generate report
        if: always()
        run: |
          npx allure generate report/allure-results

      - name: Setup Pages
        if: always()
        uses: actions/configure-pages@v3

      - name: Upload artifact
        if: always()
        uses: actions/upload-pages-artifact@v1
        with:
          path: './allure-report'

      - name: Deploy to GitHub Pages
        if: always()
        id: deployment
        uses: actions/deploy-pages@v2

Workflow Execution​

Fantastic news! Our workflow is now functioning flawlessly and exhibits complete stability. The workflow can be triggered against a single platform, such as Android or iOS, or simultaneously against both platforms in parallel.

loading...

Conclusion​

By leveraging the capabilities provided by GitHub Actions, which offer out-of-the-box SDKs for both Android and iOS, we gain a significant advantage. This allows us to construct an efficient end-to-end test pipeline without incurring any costs or relying on mobile device farm cloud services. Although conducting tests on real devices is preferable, particularly for Android, the cost-free nature of this approach presents a satisfactory compromise.

Throughout our discussion, we have provided a step-by-step demonstration of building a cross-platform end-to-end test using the GitHub Actions pipeline for native mobile Apps. We have addressed various challenges, obstacles, and issues, ensuring a thorough understanding of the process. Armed with this knowledge, you should find it easier to construct your own customized pipeline that caters to your specific requirements.

The WebdriverIO framework is a versatile tool that offers a lot of features for you to play around with. The goal of our project documentation is to communicate these features well and give you an understanding, on how they could be applied in your project. A central contributor to this are code examples. Many times they can convey the principle idea of a feature like a picture that is worth a thousand words.

It is not a surprise that many projects out there have code examples embedded in their project documentation. Many of them are even interactive and allow users to fiddle around with the code in real time, e.g. the new React Docs, or provide "playgrounds" with live examples, like on svelte.dev.

When it comes to having code examples on a documentation page, a common problem that arises with them is that examples:

• are made up and often don't reflect reality
• contain errors because we are all just humans
• are getting outdated as interfaces change
• can be difficult to apply in your own project

As an attempt to improve our code examples on this project page we started to roll out some changes to the documentation that hopefully addresses these issues:

loading...

As you can see, some examples now have two buttons that allow you to run them or view them on GitHub. But what does that mean?

Extract Examples from Docs​

As a first step we started to remove all code examples from our documentation page and moved them into a dedicated repository. This allows us to treat these examples as code and set-up the necessary infrastructure, e.g. CI/CD or automated dependency updates, to ensure quality and correctness.

So say hello 👋 to this new repository in our organization that now contains a lot of examples that we re-use on this website.

You can see that every example is self contained in its own directory to keep everything very simple. A big list of NPM scripts allows you to run specific examples with just a single command.

In order to embed the examples back into the website, we are using a plugin for Docusaurus that downloads the code based on a simple GitHub reference link. So instead of having code within our markdown files, we just reference the location on Github, e.g.:

```ts reference useHTTPS
https://github.com/webdriverio/example-recipes/blob/main/setup/testrunner.js#L5-L8
```

The plugin then downloads the code and only shows provided code lines of that file. Here is the final result of that:

loading...

If you are using a different tool for building your static docs, chances are that it has similar plugins available to just do that.

Testing Examples​

Now that we have everything nicely encapsulated into a dedicated repository, we can use CI/CD to test all examples on regular basis. A simple GitHub workflow can trigger the execution of these examples and have the pipeline fail if any of them have an error:

name: Test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        exampleDir:
          - click
          # more example directories here
          # ...
          - api/webdriver
    steps:
        - name: Checkout
          uses: actions/checkout@v3
        - uses: actions/setup-node@v3
          with:
            node-version: 18
        - name: Install
          run: npm install
        - name: Test
          run: npm run ${{ matrix.exampleDir }}
          working-directory: ${{ matrix.exampleDir }}

Most of the examples are written as normal WebdriverIO test files and contain normal assertions like any other test would do, e.g. an example that shows how to fetch elements using command chaining would be written as following:

it('should get the text of a menu link', async () => {
    const menu$ = await $('#menu') // or `browser.$('#menu')`
    console.log(await menu$.$$('li')[2].$('a').getText()) // outputs: "API"

    await expect(menu$.$$('li')[2].$('a')).toHaveText('API')
})

With the ability to reference certain code lines we can just strip out the testing part of the example and focus on what's important:

loading...

Keep Examples up to Date​

Another advantage of having a CI/CD infrastructure is the ability to use workflows that ensure everything stays up to date. Since WebdriverIO code is hosted on GitHub, we setup Dependabot to update all dependencies on weekly basis. An additional GitHub workflow helps us to auto-merge these dependency updates so that we only need to deal with them in case they cause issues due to failing tests.

This process is very important and helps us to ensure that changes in WebdriverIO don't break any of our examples we use in our documentation. It is also a great additional feedback loop and creates more confidence when a new version is released that did not break any of our examples.

Make Examples Easy Accessible​

Lastly, to make every example very easy to access and run, we are using a feature of a very cool VS Code Extension called Runme that helps to check out code locally with a simple click on a button. If you haven't installed Runme yet, go check it out on the VS Code Marketplace or find it in your VS Code:

Many surveys have shown that VS Code is the dominant IDE for many developers around the world. With the Run Example button we allow all these users to access the repository with a single click. The button is a simple link with a custom vscode:// protocol that will prompt you for permission to open it in VS Code. There, the extension will pick up the link information containing which repository it needs to check out and which markdown file to open. If the extension is not installed, it will automatically do that for you, if you consent.

Once the repository is checked out, Runme will open a dedicated README.md for the example in an interactive notebook experience. It will explain the example and walks you through it. It allows to execute the code cells within the VS Code terminal securely so that setting up and running the example is done with a simple click and requires no additional application to be opened.

For folks that don't have VS Code installed can still access the repository, check it out manually and run the examples as well.

Get Involved​

WebdriverIO has many examples and a lot of commands and APIs to document. This is a great opportunity for you to get involved and contribute to the project by:

• adding more examples to webdriverio/example-recipes or
• reference existing examples in our documentation

Feel free to also raise an issue if you have any questions or feedback.

I think this is a very cool way to provide an interactive and simple way to provide code examples for a framework that requires a specific environment that is not a browser, e.g. Node.js. If you are a framework author and run your docs with Docusaurus, feel free to copy this approach if you like it. It's Open Source and free.

Thanks for reading!

While it took a bit longer than expected the WebdriverIO team is excited to announce that we finally released v8 today! 🎉 🎉 🎉

As with almost all of the last major updates we again had to touch every single file of the project. This time our major goal for the new version was to finally transition from CommonJS to ESM which enables us to continue with important dependency updates and avoid security issues. Furthermore, we cleaned up some technical debt, e.g. removed all code related to synchronous command execution which was deprecated last year, as well as implemented a new Action API interface and streamlined the way WebdriverIO deals with global objects using the test runner.

In this blog post, we go through every important change and explain what you need to do to upgrade to v8. Spoiler alert: in most cases no updates to your tests are necessary 😉

Drop Node.js v12, v13 and v14 Support​

We've dropped support for Node v12 and v14, latter was moved into a maintenance LTS phase by the Node.js team in October 2021. While it is technically fine to continue using Node.js v14, we don't see a reason why you shouldn't update to Node.js v16 or ideally v18 directly.

To update Node.js, it is important to know how it was installed in the first place. If you are in a Docker environment, you can just upgrade the base image like:

- FROM mhart/alpine-node:14
+ FROM mhart/alpine-node:18

We recommend using NVM (Node Version Manager) to install and manage Node.js versions. You can find a detailed description of how to install NVM and update Node in their project readme.

CommonJS to ESM Transition​

The transition to the new module system has been the biggest chunk of work related to this release. It required us to update all module imports, transitioning from Jest to Vitest as a unit test framework and rewrite various parts within the code base. While this affected every single file it "should" be unrecognizable to you. If your project still uses CommonJS, WebdriverIO will work just fine as both module systems continue to be supported. This is also the case when using webdriver, devtools or webdriverio as a module.

If you have been using Babel only to use import statements in your tests, you can remove the integration as this is now supported by ESM natively. If you like to continue using CommonJS and require, that is fine too, no changes are needed to update to v8.

A new Runner for Unit and Component Testing in the Browser​

If it comes to one feature we are really excited about in this release it is the new browser runner 🙌 I've been writing and testing a lot of web components in this past year and was always frustrated about the fact that they would be tested against JSDOM rather than an actual browser. JSDOM is a re-implementation of many Web APIs in Node.js and is a great tool for simple testing but it doesn't replace an actual DOM implementation within a browser. Especially using JSDOM for component testing has various disadvantages compared to running tests in the browser.

Furthermore running component tests through WebdriverIO allows to use the WebdriverIO API seamlessly and enables real user interaction with your components through the WebDriver protocol. This makes those interactions more realistic compared to emitting them through JavaScript. It comes also with 1st class support for popular utility frameworks such as Testing Library and allows to use both APIs interchangeably. Check out how you can use Testing Library for rendering and fetching elements while using WebdriverIO for interacting with the component:

import { $, expect } from '@wdio/globals'
import { render } from '@testing-library/vue'
import Component from './components/Component.vue'

describe('Vue Component Testing', () => {
    it('increments value on click', async () => {
        // The render method returns a collection of utilities to query your component.
        const { getByText } = render(Component)

        // getByText returns the first matching node for the provided text, and
        // throws an error if no elements match or if more than one match is found.
        getByText('Times clicked: 0')

        const button = await $(getByText('increment'))

        // Dispatch a native click event to our button element.
        await button.click()
        await button.click()

        getByText('Times clicked: 2') // assert with Testing Library
        await expect($('p=Times clicked: 2')).toExist() // assert with WebdriverIO
    })
})

The new browser runner allows you to load and execute tests within the browser rather than in Node.js. This allows you to access all Web APIs to render web components or to run unit tests for your frontend modules. Under the hood it uses Vite to load all dependencies and make the integration seamless.

If you have been using Karma for running unit tests in the browser you can switch over to WebdriverIO which provides the same capabilities but offers better support and integration to other services and reporters. It also seems that the Karma project is not much maintained these days and has unresovled security vulnerabilities.

New Action Interface​

For many years users that liked to run more complex interactions on their applications using WebDriver's actions API had to know many details about the command to construct the correct payload. With v8 of WebdriverIO, we now ship a new interface that makes executing various actions much easier.

With two new browser commands: action and actions, it is now much simpler and type-safe to run the right action, e.g. sending key events to the browser:

await browser.action('key')
    .down('f').up('f')
    .down('o').up('o')
    .down('o').up('o')
    .perform()

Read more on the new action interface in the WebdriverIO API.

WebDriver BiDi Support​

A strong argument to use WebdriverIO as opposed to other tools is the fact that it is based on the WebDriver protocol, which is a web standard for automating browsers. It guarantees the ability to run tests in browsers that are used by your users as opposed to a browser engine, which can be very different from a feature and security aspect. The W3C Working Group has been working on a new version of the protocol that will enable better introspection capabilities and new automation primitives.

With this release, users can start accessing these new protocol features as they become available in browsers. Over time more commands will transition to the new protocol under the hood while the interface remains the same. We are all very excited about the new capabilities and opportunities this protocol will provide, e.g. listening to log events while running tests, e.g.:

await browser.send({
    method: 'session.subscribe',
    params: { events: ['log.entryAdded'] }
})

/**
 * returns:
 * {
 *    "method":"log.entryAdded",
 *    "params":{
 *       "type":"console",
 *       "method":"log",
 *       "realm":null,
 *       "args":[
 *          {
 *             "type":"string",
 *             "value":"Hello Bidi"
 *          }
 *       ],
 *       "level":"info",
 *       "text":"Hello Bidi",
 *       "timestamp":1657282076037
 *    }
 * }
 */
browser.on('message', (data) => console.log('received %s', data))

/**
 * trigger listener
 */
await browser.execute(() => console.log("Hello Bidi"))

We are following and supporting the development of all browser vendors to ensure new features are working as expected and can be used through a lean user interface. For more information on this topic check out my talk on The Evolution of Browser Automation.

Optional Globals​

When using the WebdriverIO test runner it would usually register the browser object or the $ and $$ command to the global scope as these are usually often used when writing tests. However, attaching objects to the global scope is not seen as best practice and can cause side effects when other modules decide to do the same. Therefore with v8, it is now up to the user whether they like to continue attaching these objects and methods to the global scope or prefer importing them directly, via:

import { browser, $, $$, expect } from '@wdio/globals'

A new configuration property called injectGlobals (defaults: true) handles whether the test runner modifies the global scope or not. If your setup works fine using global objects, no change is needed to update to v8. However, we recommend importing WebdriverIO-related interfaces directly to ensure no side effects can happen.

Note: If you are using TypeScript, updates to the tsconfig.json are necessary to reflect changes made to the location of the WebdriverIO types. These are now part of the @wdio/globals package:

{
    "compilerOptions": {
        "types": [
            "node",
-           "webdriverio/async"
+           "@wdio/globals/types"
        ]
    }
}

Miscellaneous​

Aside from these major updates, the team has spent time improving the documentation and introduced new API docs around WebdriverIO objects like browser, element and mock. Furthermore, we removed the config property from the browser object. If you have been using it to access data from the WDIO config, we suggest replacing it with options, e.g.:

- browser.config.hostname
+ browser.options.hostname

Furthermore did we fix the behavior of relative spec or exclude paths. Before v8 every path within specs, exclude or --spec was always seen relative from the users working directory. This behavior was confusing especially when the wdio.conf.js was not within the root of you project. This got fixed now so that specs and exclude path will be always seen as relative to the config file and --spec arguments, relative from the working directory.

Lastly, we had to remove support for tsconfig-paths as we haven't found a way to get it working within an ESM context. We believe this integration hasn't been used much anyway and a lot of it is nowadays natively supported in TypeScript. Let us know if this assumption is wrong and you would like to see it being supported again.

What's Next?​

The WebdriverIO team is very excited about this release as it frees up time to start working on some new cool features we put on the roadmap. For many months we have been working secretly on a VS Code Extension that makes authoring and debugging tests much easier. Aside from that, there is always plenty more work to do and opportunities to explore to make this project better. We are welcoming and supporting everyone who likes to join us.

Lastly, I would like to say thank you to everyone who supports the project. Not only the folks who contribute financially through Open Collective or Tidelift but also everyone who contributes code, ideas, reports issues or supports folks in our support chat, occasionally or on regular basis. Without contributions from the community, this project can't go anywhere. Aside from many alternative projects WebdriverIO is not funded, nor driven by any corporate interest and stays 100% community governed. No lack of funding or need for capital gains will have an impact on this project. It has been like this since its inception more than 10 years ago and will continue to be like this for many many more years. Therefore we are always looking for interested folks who like to help us hack on the project. If you haven't, join our Open Office Hours and consider giving back to the project.

I am looking forward to more years and great features ahead. Thanks for reading!