Skip to main content

Frameworks

WebdriverIO Runner has built-in support for Mocha, Jasmine, Cucumber.js, and Vitest. You can also integrate it with 3rd-party open-source frameworks, such as Serenity/JS.

Integrating WebdriverIO with test frameworks

To integrate WebdriverIO with a test framework, you need an adapter package available on NPM. Note that the adapter package must be installed in the same location where WebdriverIO is installed. So, if you installed WebdriverIO globally, be sure to install the adapter package globally, too.

Integrating WebdriverIO with a test framework lets you access the WebDriver instance using the global browser variable in your spec files or step definitions. Note that WebdriverIO will also take care of instantiating and ending the Selenium session, so you don't have to do it yourself.

Using Mocha​

First, install the adapter package from NPM:

npm install @wdio/mocha-framework --save-dev

By default WebdriverIO provides an assertion library that is built-in which you can start right away:

describe('my awesome website', () => {
    it('should do some assertions', async () => {
        await browser.url('https://webdriver.io')
        await expect(browser).toHaveTitle('WebdriverIO · Next-gen browser and mobile automation test framework for Node.js | WebdriverIO')
    })
})

WebdriverIO v10 ships Mocha 12 and supports Mocha's BDD (default), TDD, and QUnit interfaces.

If you like to write your specs in TDD style, set the ui property in your mochaOpts config to tdd. Now your test files should be written like this:

suite('my awesome website', () => {
    test('should do some assertions', async () => {
        await browser.url('https://webdriver.io')
        await expect(browser).toHaveTitle('WebdriverIO · Next-gen browser and mobile automation test framework for Node.js | WebdriverIO')
    })
})

If you want to define other Mocha-specific settings, you can do it with the mochaOpts key in your configuration file. A list of all options can be found on the Mocha project website.

Note: WebdriverIO does not support the deprecated usage of done callbacks in Mocha:

it('should test something', (done) => {
    done() // throws "done is not a function"
})

Mocha Options​

The following options can be applied in your wdio.conf.js to configure your Mocha environment. Note: not all Mocha options are supported. parallel still belongs to Mocha's own worker pool and will error here — the WDIO testrunner already parallelizes specs across capabilities and workers. To run the tests of one spec file at the same time, use parallelMode. Mocha 12's CLI also moved off yargs onto Node's util.parseArgs; that only affects a direct mocha invocation, not mochaOpts passed through wdio. You can pass these framework options as arguments, e.g.:

wdio run wdio.conf.ts --mochaOpts.grep "my test" --mochaOpts.bail --no-mochaOpts.checkLeaks

This will pass along the following Mocha options:

{
    grep: ['my-test'],
    bail: true
    checkLeacks: false
}

The following Mocha options are supported:

require​

Type
string|string[]
Default
[]

The require option is useful when you want to add or extend some basic functionality (WebdriverIO framework option).

allowUncaught​

Type
boolean
Default
false

Propagate uncaught errors.

bail​

Type
boolean
Default
false

Bail after first test failure.

checkLeaks​

Type
boolean
Default
false

Check for global variable leaks.

delay​

Type
boolean
Default
false

Delay root suite execution.

failHookAffectedTests​

Type
boolean
Default
true

Report each test skipped by a failing before or beforeEach hook as a failure. WebdriverIO enables this so a broken setup hook is visible on every spec it skipped. Set it to false to report only the hook.

fgrep​

Type
string
Default
null

Test filter given string.

forbidOnly​

Type
boolean
Default
false

Tests marked only fail the suite.

forbidPending​

Type
boolean
Default
false

Pending tests fail the suite.

fullTrace​

Type
boolean
Default
false

Full stacktrace upon failure.

global​

Type
string[]
Default
[]

Variables expected in global scope.

grep​

Type
RegExp|string
Default
null

Test filter given regular expression. Mocha 12 accepts modern RegExp flags in this filter (for example s or d).

invert​

Type
boolean
Default
false

Invert test filter matches.

maxParallelContexts​

Type
number
Default
os.cpus().length

Number of tests that run at the same time when parallelMode is set. See Parallel Tests in a Spec File.

parallelMode​

Type
'contexts' | 'userContexts'
Default
undefined

Run the tests of a spec file at the same time, each in its own WebDriver BiDi browsing context. 'contexts' shares cookies and storage between tests, 'userContexts' isolates them. Requires the 'wdio:experimentalBiDiCommands': true capability. bail and retries are not supported in this mode. See Parallel Tests in a Spec File.

retries​

Type
number
Default
0

Number of times to retry failed tests.

timeout​

Type
number
Default
30000

Timeout threshold value (in ms).

Using Jasmine​

First, install the adapter package from NPM:

npm install @wdio/jasmine-framework --save-dev

You can then configure your Jasmine environment by setting a jasmineOpts property in your config. A list of all options can be found on the Jasmine project website.

Jasmine Options​

The following options can be applied in your wdio.conf.js to configure your Jasmine environment using the jasmineOpts property. For more information on these configuration options, check out the Jasmine docs. You can pass these framework options as arguments, e.g.:

wdio run wdio.conf.ts --jasmineOpts.grep "my test" --jasmineOpts.failSpecWithNoExpectations --no-jasmineOpts.random

This will pass along the following Jasmine options:

{
    grep: 'my test',
    failSpecWithNoExpectations: true,
    random: false
}

The following Jasmine options are supported:

defaultTimeoutInterval​

Type
number
Default
60000

Default Timeout Interval for Jasmine operations.

helpers​

Type
string[]
Default
[]

Array of filepaths (and globs) relative to spec_dir to include before jasmine specs.

requires​

Type
string[]
Default
[]

The requires option is useful when you want to add or extend some basic functionality.

random​

Type
boolean
Default
false

Whether to randomize spec execution order. Jasmine's own default is true, but WebdriverIO runs the specs in order unless you set this option.

seed​

Type
Function
Default
null

Seed to use as the basis of randomization. Null causes the seed to be determined randomly at the start of execution.

failSpecWithNoExpectations​

Type
boolean
Default
false

Whether to fail the spec if it ran no expectations. By default a spec that ran no expectations is reported as passed. Setting this to true will report such spec as a failure.

oneFailurePerSpec​

Type
boolean
Default
false

Stop a spec at its first failed expectation. A failed sync matcher stops the spec at once, and an awaited async matcher stops it when its promise settles. The other specs continue to run.

specFilter​

Type
Function
Default
(spec) => true

Function to use to filter specs.

grep​

Type
string|Regexp
Default
null

Only run tests matching this string or regexp. (Only applicable if no custom specFilter function is set)

invertGrep​

Type
boolean
Default
false

If true it inverts the matching tests and only runs tests that don't match with the expression used in grep. (Only applicable if no custom specFilter function is set)

stopOnSpecFailure​

Type
boolean
Default
false

Stop the spec file at its first failed spec (it): the other specs of the file do not run, also in other describe blocks. Other spec files run in their own workers and continue.

cleanStack​

Type
boolean
Default
true

Remove the lines of node_modules packages from the stack traces of failures.

expectationResultHandler​

Type
Function
Default
null

Called with (passed, assertion) for each expectation, for example to take a screenshot when an expectation fails. If the function throws for a passed expectation, the expectation fails with that error.

Assertions​

With Jasmine, the global expect combines Jasmine's matchers and the WebdriverIO matchers:

  • Jasmine's matchers (toBe, toEqual, toHaveBeenCalled, …) and the matchers that you add with jasmine.addMatchers are synchronous. They return undefined, so you do not need await.
  • WebdriverIO matchers, Jasmine's async matchers (toBeResolved, toBeRejectedWith, …) and the matchers that you add with jasmine.addAsyncMatchers return a promise. Always await them.

Use expect() for both kinds: it sends each matcher to Jasmine's expect or expectAsync for you. await expectAsync($('#logo')).toBeDisplayed() also works. For TypeScript, @wdio/jasmine-framework in types also gives expectAsync() the WebdriverIO matchers.

it('checks the page', async () => {
    expect([1, 2]).toHaveSize(2)                                   // Jasmine, sync
    await expect($('#logo')).toHaveSize({ width: 32, height: 32 }) // WebdriverIO, async
    await expect(loadData()).toBeResolved()                        // Jasmine async matcher
})

toHaveSize exists in both libraries. The WebdriverIO matcher runs on WebdriverIO values: an element, an element array or Element[] (for example the result of $$().filter()), a multi-remote element, a browser, a browsing context, a mock, the some() wrapper, or a promise such as a chainable $(). Jasmine's matcher runs on every other value.

The asymmetric matchers of both libraries work, in Jasmine and in WebdriverIO matchers: jasmine.any(), jasmine.objectContaining(), jasmine.stringMatching(), … and expect.any(), expect.stringContaining(), expect.oneOf(), expect.multiRemote(), expect.not.stringContaining(), …. To use some(), import it:

import { some } from 'expect-webdriverio/api'

await expect(some($$('li'))).toHaveAttribute('data-state', 'on')

The Jest parts of expect are not available with Jasmine: Jest-only matchers such as toStrictEqual or toHaveLength, and expect.soft(). To add a custom matcher, use expect.extend() in a spec file or the before hook (see Custom Matchers), or jasmine.addMatchers for a sync matcher and jasmine.addAsyncMatchers for an async matcher.

For TypeScript, add jasmine to types, see TypeScript Setup.

Using Cucumber​

First, install the adapter package from NPM:

npm install @wdio/cucumber-framework --save-dev

If you want to use Cucumber, set the framework property to cucumber by adding framework: 'cucumber' to the config file .

Options for Cucumber can be given in the config file with cucumberOpts. Check out the whole list of options here. The adapter uses Cucumber 13. tagExpression has been removed; filter with tags. See the v10 migration guide.

To get up and running quickly with Cucumber, have a look on our cucumber-boilerplate project that comes with all the step definitions you need to get stared, and you'll be writing feature files right away.

Cucumber Options​

The following options can be applied in your wdio.conf.js to configure your Cucumber environment using the cucumberOpts property:

Adjusting options through the command line

The cucumberOpts, such as custom tags for filtering tests, can be specified through the command line. This is accomplished by using the cucumberOpts.{optionName}="value" format.

For example, if you want to run only the tests that are tagged with @smoke, you can use the following command:

# When you only want to run tests that hold the tag "@smoke"
npx wdio run ./wdio.conf.js --cucumberOpts.tags="@smoke"
npx wdio run ./wdio.conf.js --cucumberOpts.name="some scenario name" --cucumberOpts.failFast

This command sets the tags option in cucumberOpts to @smoke, ensuring that only tests with this tag are executed.

backtrace​

Type
Boolean
Default
false

Show full backtrace for errors.

requireModule​

Type
string[]
Default
[]

Require modules prior to requiring any support files.

Example:

cucumberOpts: {
    requireModule: ['@babel/register']
    // or
    requireModule: [
        [
            '@babel/register',
            {
                rootMode: 'upward',
                ignore: ['node_modules']
            }
        ]
    ]
 }

failFast​

Type
boolean
Default
false

Abort the run on first failure.

name​

Type
RegExp[]
Default
[]

Only execute the scenarios with name matching the expression (repeatable).

require​

Type
string[]
Default
[]

Require files containing your step definitions before executing features. You can also specify a glob to your step definitions.

Example:

cucumberOpts: {
    require: [path.join(__dirname, 'step-definitions', 'my-steps.js')]
}

import​

Type
String[]
Default
[]

Paths to where your support code is, for ESM.

Example:

cucumberOpts: {
    import: [path.join(__dirname, 'step-definitions', 'my-steps.js')]
}

strict​

Type
boolean
Default
true

Fail the run on a pending step. The default is true, as in Cucumber. The reporters still show the step as pending (Cucumber shows it as a warning), and the worker logs a warning that names the scenarios with a pending step or hook. An undefined step fails the run also without strict, unless ignoreUndefinedDefinitions is true.

tags​

Type
String
Default
""

Only execute the features or scenarios with tags matching the expression. A feature file with no matching scenario is reported as skipped and starts no browser session. Please see the Cucumber documentation for more details.

timeout​

Type
Number
Default
60000

Timeout in milliseconds for step definitions.

retry​

Type
Number
Default
0

Specify the number of times to retry failing test cases.

retryTagFilter​

Type
String

Only retry the scenarios with tags matching the tag expression. Requires retry.

language​

Type
String
Default
en

Default language for your feature files

order​

Type
String
Default
defined

Run the scenarios in the order of the feature files ('defined') or in a random order ('random'). With 'random:<seed>', for example 'random:42', a random order is the same in each run. With 'random', the log shows the seed of the run (Random order using seed: 42).

format​

Type
string[]

Name and output file path of formatter to use. WebdriverIO primarily supports only the Formatters that writes output to a file. Formatters don't run with parallelMode.

formatOptions​

Type
object

Options to be provided to formatters

tagsInTitle​

Type
Boolean
Default
false

Add cucumber tags to feature or scenario name

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

ignoreUndefinedDefinitions​

Type
Boolean
Default
false

Report an undefined step as pending, with (undefined step) at the end of its title, and do not fail the run for it.

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

failAmbiguousDefinitions​

Type
Boolean
Default
false

Report an ambiguous step (a step that matches more than one step definition) as failed, with the list of the matching step definitions. With false, the step is reported as skipped, with the list as its reason. In both cases, the run fails. Exception: without parallelMode, the run passes with ignoreUndefinedDefinitions: true and failAmbiguousDefinitions: false.

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

profiles​

Type
string[]
Default
[]

The profiles of the Cucumber configuration file to use.

Cucumber merges the values in this order: Cucumber's defaults, then the profiles or the configuration file (without file, Cucumber's default file in the working folder, for example cucumber.js), then cucumberOpts. So a value in cucumberOpts wins over the profile, and a profile value wins over Cucumber's default. tags and retryTagFilter of both are joined with and, and lists (for example require, import, name) are added together. A folder in the require or import of a profile is not loaded (use a glob). The paths, requireModule and loader of a profile or configuration file are not used: the feature files come from specs, the modules from cucumberOpts.requireModule, and the worker logs a warning. parallel and shard of a profile are not used either (WebdriverIO runs the workers), and a profile does not turn on publish: set it in cucumberOpts.

file​

Type
string | false
Default
undefined

Path of the Cucumber configuration file to load (absolute, or from the working folder), or false to load none. Without it, Cucumber looks for its default configuration files.

dryRun​

Type
Boolean
Default
false

Run no step and no hook: Cucumber only matches each step with its step definitions, so undefined and ambiguous steps are still reported. The run does not fail.

worldParameters​

Type
object
Default
{}

Parameters for your World, as this.parameters.

scenarioLevelReporter​

Type
Boolean
Default
false

Report each scenario as one test. Its steps and hooks are not reported. A BeforeAll / AfterAll is still reported, as a hook of the feature.

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

parallelMode​

Type
'contexts' | 'userContexts'
Default
undefined

Run the scenarios of a feature file at the same time, each in its own WebDriver BiDi browsing context. 'contexts' shares cookies and storage between scenarios, 'userContexts' isolates them. Requires the 'wdio:experimentalBiDiCommands': true capability. See Parallel Tests in a Spec File.

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

maxParallelContexts​

Type
number
Default
os.cpus().length

Number of scenarios that run at the same time when parallelMode is set. See Parallel Tests in a Spec File.

Please note that this is a @wdio/cucumber-framework specific option and not recognized by cucumber-js itself

Skipping tests in cucumber​

Note that if you want to skip a test using regular cucumber test filtering capabilities available in cucumberOpts, you will do it for all the browsers and devices configured in the capabilities. In order to be able to skip scenarios only for specific capabilities combinations without having a session started if not necessary, webdriverio provides the following specific tag syntax for cucumber:

@skip([condition])

were condition is an optional combination of capabilities properties with their values that when all matched with cause the tagged scenario or feature to be skipped. Of course you can add several tags to scenarios and features to skip a tests under several different conditions.

You can also use the '@skip' annotation to skip tests without changing tags. In this case the skipped tests will be displayed in the test report.

Here you have some examples of this syntax:

  • @skip or @skip(): will always skip the tagged item
  • @skip(browserName="chrome"): the test will not be executed against chrome browsers.
  • @skip(browserName="firefox";platformName="linux"): will skip the test in firefox over linux executions.
  • @skip(browserName=["chrome","firefox"]): tagged items will be skipped for both chrome and firefox browsers.
  • @skip(browserName=/i.*explorer/): capabilities with browsers matching the regexp will be skipped (like iexplorer, internet explorer, internet-explorer, ...).

Import Step Definition Helper​

In order to use step definition helper like Given, When or Then or hooks, you are suppose to import then from @cucumber/cucumber, e.g. like this:

import { Given, When, Then } from '@cucumber/cucumber'

Now, if you use Cucumber already for other types of tests unrelated to WebdriverIO for which you use a specific version you need to import these helpers in your e2e tests from the WebdriverIO Cucumber package, e.g.:

import { Given, When, Then, world, context } from '@wdio/cucumber-framework'

This ensures that you use the right helpers within the WebdriverIO framework and allows you to use an independent Cucumber version for other types of testing.

Publishing Report​

Cucumber provides a feature to publish your test run reports to https://reports.cucumber.io/, which can be controlled either by setting the publish flag in cucumberOpts or by configuring the CUCUMBER_PUBLISH_TOKEN environment variable. However, when you use WebdriverIO for test execution, there's a limitation with this approach. It updates the reports separately for each feature file, making it difficult to view a consolidated report.

To overcome this limitation, we've introduced a promise-based method called publishCucumberReport within @wdio/cucumber-framework. This method should be called in the onComplete hook, which is the optimal place to invoke it. publishCucumberReport requires the input of the report directory where cucumber message reports are stored.

You can generate cucumber message reports by configuring the format option in your cucumberOpts. It's highly recommended to provide a dynamic file name within the cucumber message format option to prevent overwriting reports and ensure that each test run is accurately recorded.

Before using this function, make sure to set the following environment variables:

  • CUCUMBER_PUBLISH_REPORT_URL: The URL where you want to publish the Cucumber report. If not provided, the default URL 'https://messages.cucumber.io/api/reports' will be used.
  • CUCUMBER_PUBLISH_REPORT_TOKEN: The authorization token required to publish the report. If this token is not set, the function will exit without publishing the report.

Here's an example of the necessary configurations and code samples for implementation:

import { v4 as uuidv4 } from 'uuid'
import { publishCucumberReport } from '@wdio/cucumber-framework';

export const config = {
    // ... Other Configuration Options
    cucumberOpts: {
        // ... Cucumber Options Configuration
        format: [
            ['message', `./reports/${uuidv4()}.ndjson`],
            ['json', './reports/test-report.json']
        ]
    },
    async onComplete() {
        await publishCucumberReport('./reports');
    }
}

Please note that ./reports/ is the directory where cucumber message reports will be stored.

Using Vitest​

@wdio/vitest-framework runs specs written in Vitest syntax (describe, test, test.concurrent, test.extend) through the WDIO testrunner. Vitest schedules the tests, so the testrunner starts no WDIO workers. wdio.conf.ts stays the source of truth for specs, capabilities, connection options and hooks. Use it when you want Vitest's API, especially fixtures and test.concurrent. Otherwise Mocha is the default choice.

First, install the adapter package and vitest from NPM. vitest is a peer dependency, so your project and the adapter share one copy. expect-webdriverio is installed with the adapter:

npm install @wdio/vitest-framework vitest --save-dev

Then set framework: 'vitest' in your config file:

wdio.conf.ts
export const config: WebdriverIO.Config = {
    framework: 'vitest',
    specs: ['./test/specs/**/*.spec.ts'],
    baseUrl: 'https://the-internet.herokuapp.com',
    capabilities: [{ browserName: 'chrome', webSocketUrl: true, 'wdio:experimentalBiDiCommands': true }],
    vitestOpts: {
        parallelMode: 'userContexts'
    }
}

Run it like any other WDIO project:

npx wdio run wdio.conf.ts

Vitest globals are enabled at runtime, and browser, driver, $, $$ and expect are globals as well. Import describe, test and the hooks from vitest, so TypeScript knows their types:

test/specs/login.spec.ts
import { describe, test } from 'vitest'

describe('login page', () => {
    test('shows the form', async () => {
        await browser.url('/login')
        await expect($('#login')).toBeDisplayed()
    })
})

The global expect is the WebdriverIO assertion library (expect-webdriverio), which replaces Vitest's own expect. waitforTimeout and waitforInterval from the config set its defaults. Never import { expect } from 'vitest'. That expect has no WebdriverIO matchers such as toBeDisplayed.

For TypeScript, add @wdio/vitest-framework to types so vitestOpts is typed. It also provides the Vitest globals (describe, test and the hooks) without imports, see TypeScript Setup:

tsconfig.json
{
    "compilerOptions": {
        "types": ["node", "@wdio/globals/types", "@wdio/vitest-framework"]
    }
}

How Vitest runs your specs​

  • Every capability becomes one Vitest project, named <browserName>-<index>, for example chrome-0.
  • Every spec file gets its own browser session. The session starts in beforeAll and is deleted in afterAll.
  • The WDIO hooks beforeSession, before, after and afterSession run once per spec file, and beforeTest and afterTest once per test. Worker services (for example Sauce Labs, BrowserStack or Appium) are initialized like in the WDIO runner, so their hooks run too.
  • The console output comes from Vitest. File reporters from wdio.conf.ts, such as junit, allure or json, receive the Vitest results. Every spec file and capability is reported as one job. Skipped and todo tests are reported as pending.
  • maxInstances (or wdio:maxInstances per capability) sets Vitest's maxWorkers, and bail sets Vitest's bail.
  • runnerEnv is passed to the Vitest workers as Vitest's env, with NODE_ENV defaulting to test. vitestOpts.env is merged on top and wins. NODE_OPTIONS in runnerEnv only reaches the worker's process.env, because Vitest has already forked the worker. Node flags such as --import do not apply to it.

Concurrent tests​

test.concurrent runs the tests of one spec file at the same time, each in its own browser window (browsing context). Every command in the test, including the ones in your page objects and fixtures, targets that window. The window is closed when the test ends, also when it fails. This needs parallelMode, a WebDriver BiDi session (webSocketUrl: true) and the 'wdio:experimentalBiDiCommands': true capability. Without that capability, commands use classic WebDriver and cannot target the test's window. It changes how element commands are implemented in every test of the session, see Behavior changes of the BiDi element commands.

test/specs/checkout.spec.ts
import { describe, test } from 'vitest'

describe('checkout', () => {
    test.concurrent('guest', async () => {
        await browser.url('/checkout')
        await expect($('h1')).toHaveText('Checkout')
    })

    test.concurrent('member', async () => {
        await browser.url('/checkout?member=1')
        await expect($('h1')).toHaveText('Checkout')
    })
})

Without parallelMode, test.concurrent fails with:

test.concurrent needs vitestOpts.parallelMode ('contexts' | 'userContexts'): concurrent tests would share one window

Without the capability, it fails with:

test.concurrent with vitestOpts.parallelMode needs the 'wdio:experimentalBiDiCommands': true capability, so commands run in the window of their test

Vitest Options​

Set them in vitestOpts in your wdio.conf.ts.

parallelMode​

Runs test.concurrent tests at the same time, each in its own browsing context. Requires a WebDriver BiDi session and the 'wdio:experimentalBiDiCommands': true capability.

  • 'contexts': windows share the default user context, so cookies, localStorage and service workers are shared across concurrent tests.
  • 'userContexts': each test gets its own user context, so storage is isolated per test. Pick this when tests log in or change cookies. browser.mock() remains session-wide in both modes.

Type: 'contexts' | 'userContexts'
Default: undefined

maxParallelContexts​

Maximum number of test.concurrent tests that run at once. Maps to Vitest's maxConcurrency and only applies when parallelMode is set.

Type: number
Default: os.cpus().length

Other Vitest options​

All other keys are passed to Vitest's test config and override this framework's defaults, for example testTimeout (default 60000), hookTimeout (default: connectionRetryTimeout, else 120000, because beforeAll starts the session), retry or reporters. With retry, every attempt is reported to the WDIO reporters: the failed ones as retries, the last one as the test result.

The framework controls some keys itself:

  • include is always the specs resolved from wdio.conf.ts (respecting --spec and exclude), so vitestOpts.include is ignored.
  • pool is always 'forks'.
  • setupFiles from vitestOpts runs after the framework's own setup file instead of replacing it.
  • A vitest.config.ts file is not read. Put Vitest options in vitestOpts.

Limitations​

  • Multi-remote capabilities are not supported yet.
  • expect.poll is not available, because the global expect comes from expect-webdriverio.
  • switchWindow and switchFrame throw in a WebDriver BiDi session, which parallelMode requires. Inside concurrent tests, getWindowSize, setWindowSize, throttleCPU, throttleNetwork and classic window and frame protocol commands such as switchToWindow also throw, because they change session-wide state.
  • The hooks beforeSuite, afterSuite, beforeHook, afterHook, beforeAssertion, afterAssertion, onWorkerStart and onWorkerEnd are not run, in wdio.conf.ts or in services.
  • hook:start / hook:end events are sent per hook kind and attempt, not per hook: one pair for all beforeEach hooks and one for all afterEach hooks, with no duration. A failed hook ends with state: 'fail' and its error. A failing beforeAll / afterAll is reported separately, see below. Reporter runtime API calls made inside a beforeEach / afterEach hook are attributed to the test, not to the hook.
  • --watch, --repeat and per-capability wdio:specs / wdio:exclude are not supported yet. The testrunner stops with an error instead of ignoring them.
  • WebDriver commands are not sent to the WDIO reporters, because the reporters run in the launcher and the commands run in the Vitest workers. Allure therefore shows no WebDriver steps and no automatic screenshots (disableWebdriverStepsReporting and disableWebdriverScreenshotsReporting have no effect), the video reporter records nothing, and the HTML reporter shows no automatic screenshots. Attach screenshots explicitly, for example allure.addAttachment('screenshot', Buffer.from(await browser.takeScreenshot(), 'base64'), 'image/png') in afterTest or afterEach. The beforeCommand / afterCommand hooks of wdio.conf.ts and of worker services still run. Calls of the HTML reporter (ReportEvents.logMessage, logScreenshot and videoCapture) are relayed, so their logs, screenshots and videos reach the report; WebDriver commands still do not.
  • Top-level config changes made in onPrepare do not reach the specs, because each Vitest worker parses the config file again. Changes to capabilities do reach them.
  • Reporter runtime APIs such as addStep and addAttachment from @wdio/allure-reporter or addProperty from @wdio/junit-reporter work from tests and beforeEach / afterEach hooks. Calls in beforeAll / afterAll are dropped. With exactly one Allure reporter in the final reporter list (after onPrepare ran), the launcher tells the Vitest workers its results folder, they write Allure attachments there and only the path travels to the launcher. Otherwise attachments are sent through the launcher.
  • A failing beforeAll or afterAll is reported to WDIO reporters as a failed hook, so the JUnit reporter adds a failing testcase for it. Other beforeAll / afterAll results are not reported.
  • Soft assertions (expect.soft from expect-webdriverio) fail the test at its end unless autoAssertOnTestEnd is false. expect-webdriverio tracks one current test per worker, so in test.concurrent tests the failures can be attributed to the wrong test. Use Vitest's own expect.soft from the test context there.
  • Soft assertions outside a test (in beforeAll or afterAll) are not asserted: their failures are dropped.

Using Serenity/JS​

Serenity/JS is an open-source framework designed to make acceptance and regression testing of complex software systems faster, more collaborative, and easier to scale.

For WebdriverIO test suites, Serenity/JS offers:

Serenity BDD Report Example

Installing Serenity/JS​

To add Serenity/JS to an existing WebdriverIO project, install the following Serenity/JS modules from NPM:

npm install @serenity-js/{core,web,webdriverio,assertions,console-reporter,serenity-bdd} --save-dev

Learn more about Serenity/JS modules:

Configuring Serenity/JS​

To enable integration with Serenity/JS, configure WebdriverIO as follows:

wdio.conf.ts
import { WebdriverIOConfig } from '@serenity-js/webdriverio';

export const config: WebdriverIOConfig = {

    // Tell WebdriverIO to use Serenity/JS framework
    framework: '@serenity-js/webdriverio',

    // Serenity/JS configuration
    serenity: {
        // Configure Serenity/JS to use the appropriate adapter for your test runner
        runner: 'cucumber',
        // runner: 'mocha',
        // runner: 'jasmine',

        // Register Serenity/JS reporting services, a.k.a. the "stage crew"
        crew: [
            // Optional, print test execution results to standard output
            '@serenity-js/console-reporter',

            // Optional, produce Serenity BDD reports and living documentation (HTML)
            '@serenity-js/serenity-bdd',
            [ '@serenity-js/core:ArtifactArchiver', { outputDirectory: 'target/site/serenity' } ],

            // Optional, automatically capture screenshots upon interaction failure
            [ '@serenity-js/web:Photographer', { strategy: 'TakePhotosOfFailures' } ],
        ]
    },

    // Configure your Cucumber runner
    cucumberOpts: {
        // see Cucumber configuration options below
    },

    // ... or Jasmine runner
    jasmineOpts: {
        // see Jasmine configuration options below
    },

    // ... or Mocha runner
    mochaOpts: {
        // see Mocha configuration options below
    },

    runner: 'local',

    // Any other WebdriverIO configuration
};

Learn more about:

Producing Serenity BDD reports and living documentation​

Serenity BDD reports and living documentation are generated by Serenity BDD CLI, a Java program downloaded and managed by the @serenity-js/serenity-bdd module.

To produce Serenity BDD reports, your test suite must:

  • download the Serenity BDD CLI, by calling serenity-bdd update which caches the CLI jar locally
  • produce intermediate Serenity BDD .json reports, by registering SerenityBDDReporter as per the configuration instructions
  • invoke the Serenity BDD CLI when you want to produce the report, by calling serenity-bdd run

The pattern used by all the Serenity/JS Project Templates relies on using:

  • a postinstall NPM script to download the Serenity BDD CLI
  • npm-failsafe to run the reporting process even if the test suite itself has failed (which is precisely when you need test reports the most...).
  • rimraf as a convenience method to remove any test reports left over from the previous run
package.json
{
  "scripts": {
    "postinstall": "serenity-bdd update",
    "clean": "rimraf target",
    "test": "failsafe clean test:execute test:report",
    "test:execute": "wdio wdio.conf.ts",
    "test:report": "serenity-bdd run"
  }
}

To learn more about the SerenityBDDReporter, please consult:

Using Serenity/JS Screenplay Pattern APIs​

The Screenplay Pattern is an innovative, user-centred approach to writing high-quality automated acceptance tests. It steers you towards an effective use of layers of abstraction, helps your test scenarios capture the business vernacular of your domain, and encourages good testing and software engineering habits on your team.

By default, when you register @serenity-js/webdriverio as your WebdriverIO framework, Serenity/JS configures a default cast of actors, where every actor can:

This should be enough to help you get started with introducing test scenarios that follow the Screenplay Pattern even to an existing test suite, for example:

specs/example.spec.ts
import { actorCalled } from '@serenity-js/core'
import { Navigate, Page } from '@serenity-js/web'
import { Ensure, equals } from '@serenity-js/assertions'

describe('My awesome website', () => {
    it('can have test scenarios that follow the Screenplay Pattern', async () => {
        await actorCalled('Alice').attemptsTo(
            Navigate.to(`https://webdriver.io`),
            Ensure.that(
                Page.current().title(),
                equals(`WebdriverIO · Next-gen browser and mobile automation test framework for Node.js | WebdriverIO`)
            ),
        )
    })

    it('can have non-Screenplay scenarios too', async () => {
        await browser.url('https://webdriver.io')
        await expect(browser)
            .toHaveTitle('WebdriverIO · Next-gen browser and mobile automation test framework for Node.js | WebdriverIO')
    })
})

To learn more about the Screenplay Pattern, check out:

Welcome! How can I help?

WebdriverIO AI Copilot