For many years one of the selling features of the WebdriverIO framework was its synchronous API. Especially for folks coming from more synchronous oriented languages such as Java or Ruby, it has helped to avoid race conditions when executing commands. But also people that are more familiar with Promises tend to prefer synchronous execution as it made the code easier to read and handle.

Running asynchronous commands in a synchronous way was possible with the help of the @wdio/sync package. If installed, WebdriverIO would automatically wrap the commands with utility methods that were using the fibers package to create a synchronous execution environment. It uses some internal V8 APIs to allow to jump between multiple call stacks from within a single thread. This approach also has been popular among other projects, e.g. Meteor, where most of the code is written using asynchronous APIs which causes developers to constantly start the line of code with await.

Last year the author of the Fibers package announced that he would no longer continue to maintain the project anymore. He built Fibers when JavaScript did not have any proper mechanism to handle asynchronous code other than using callbacks. With JavaScript evolving and adding APIs like Promises or Generators there is technically no reason anymore for Fibers to exist other than preference of code style. Now with the release of Node.js v16 and the update to V8 v9 Fibers stopped working due to a change in V8 that would remove some internal interfaces Fibers was using. Given that a fix for this is non trivial and the maintainer already stepped down from the project it is unlikely that we will see support for Fibers in Node.js v16 and on.

After the WebdriverIO team discovered this we immediately took action and evaluated our options. We opened an RFC to discuss with the community in which direction the project should go. I would like to thank everyone who chimed in and provided their opinion. We experimented with some alternative options, e.g. using Babel to transpile synchronous code into asynchronous but they all failed due to various reasons. The ultimate decision was made to accept the fact that synchronous command execution won't be possible moving on and rather embrace asynchronicity.

With the release of WebdriverIO v7.9 we are happy to announce that we improved our asynchronous API and matched it with the synchronous one. When chaining element command calls users had to write aweful code like this before:

await (await (await $('#foo')).$$('.bar'))[42].click()

now this got simiplified to this:

await $('#foo').$$('.bar')[42].click()

Thanks to the enormous power of the Proxy Object the API is now much more streamlined and less verbose. This will also help users migrating a project that uses the synchronous API to become asynchronous. The team will work on a codemod to help make this process as automated and easy as possible.

At this point the WebdriverIO team wants to thank Marcel Laverdet (@laverdet on GitHub) for building Fibers and maintaining it for so many years. It is time to move on and embrace all the great JavaScript language feature many people have worked hard on. While we have updated the code examples in our docs, @wdio/sync will continue to be supported until we drop support for Node.js v14 and earlier which won't happen before April 2023. You will have enough time to slowly migrate your tests using async/await.

If you have any questions on this or the migration from a framework writing with synchronous commands to asynchronous code, feel free to drop us a line in our discussion forum or on our community Discord support server.

Choosing WebdriverIO​

JW Player is an embeddable, online video player which generates over a billion unique views every day. In order to sustain and grow this scale, the player needs to be able to function on a multitude of different web and mobile platforms. This increases the importance of automated testing to improve confidence in our releases when deploying to so many different targets. After a lengthy project of converting our legacy test framework, which comprised over 6,000 tests, the Test Engineering team at JW Player has been able to deliver more timely releases with fewer regressions. We have experienced no major rollbacks, and increased the confidence we have in the quality of our own product, thanks to WebdriverIO.

Before our migration to WebdriverIO, we had been using an open sourced Ruby framework on top of Cucumber. JW Player is officially supported on seven desktop and mobile web browsers, as well as iOS and Android versions dating back to 10 and 4.4, respectively. For coverage of these platforms, we run approximately 25,000 UI acceptance tests on a nightly basis. The legacy implementation created two problems. First, we encountered performance limitations in Ruby, as a single test run across all platforms could take up to 9 hours. Second, as the Player is implemented in JavaScript, product engineers were less likely to embrace and contribute to the Ruby-based framework. Moving to a JavaScript-native framework addressed both of these problems.

Selenium Webdriver has long been the go-to standard for web automation. Around 2018, our team began to explore several new emerging testing technologies. Cypress had limited browser support, Microsoft Playwright had not yet even been released, and Puppeteer would only execute on Chrome. A Webdriver-based solution, with its broad and dedicated support amongst browser vendors, was the clear winner.

What initially attracted us to WebdriverIO was its straightforward API and complete support for all browsers and devices we needed to test, compared to Cypress and Puppeteer which lack support for one or more of the necessary platforms. More importantly, though, was its rich plugin system and active, engaged developer community. Sponsorship from Sauce Labs, which had already made a name for itself within the testing space, gave us confidence that WebdriverIO would continue to grow and not become abandonware.

Out of the box, WebdriverIO had support for several of our existing and desired toolsets. Tools such as Allure reporting, which we use to quickly comb through product defects, as well as Report Portal, which we use to monitor test health and track trends over time, were easy for us to integrate. The granular pre and post execution hooks gave the test engineers an unprecedented ability to shape how and where tests executed.

Webdriver.IO Practical Approach​

As more features keep being added to WebdriverIO, we are continually able to simplify our codebase by removing open source dependencies and messy mixin code. We have even been able to decommission services that our old framework relied on altogether.

• The Network Primitives feature, released last year, allowed us to remove our dependency on Browsermob Proxy, a proxy tool commonly used in Selenium Webdriver applications to intercept and manipulate HTTP requests. We now call browser.mock(), specify a substring or regex of the request we want to capture and supply a simple mock object to replace the asset. The ability to delay a response allowed us to automate several complicated tests which had required manual execution. We were then able to validate the player’s behavior when particular requests, such as ads, are delayed due to network or other external conditions:

  // mock.js
  export function delayResponse3seconds() {
      return new Promise((resolve) => setTimeout(() => {
          return resolve('{ "foo": bar }');
      }, 3000));
  }
  
  // test.js
  import { delayResponse3seconds } from './mock';
  
  function rewritePattern(pattern, replacement) {
     console.log(`Attempting to rewrite: ${pattern} with: ${replacement}`);
     const toRewrite = browser.mock(`**/${pattern}`);
     toRewrite.respond(delayResponse3seconds);
     console.log(`Successfully rewrote ${pattern} to ${replacement}`);
  }
  

• The Chrome Devtools Protocol support that comes out of the box has also enabled us to automate some tests that were a manual chore. Being able to call browser.throttle(“Good3G”) after our initial page load has allowed us to more accurately verify how the video player behaves under more real world conditions for our mobile users.

• Thanks to WebdriverIO’s CDP mapping, we have been able to create and maintain a suite of performance tests. Calling browser.getMetrics() after a page load and interacting with the player has enabled us to verify that once a player is set up and embedded onto a customer’s website, it will not cause any undue Cumulative Layout Shifts for the end user, which create a disruptive page loading experience.

Summary​

Overall, JW Player’s migration to WebdriverIO was nothing short of a huge success. Between the performance and “quality of life” improvements over our old framework, WebdriverIO’s feature set has allowed us to automate a couple of hundred manual test cases. We've been able to greatly cut down the length of our regression cycles from approximately 1 week to just a couple of days. Most importantly, however, these improvements have allowed us to find a record number of defects, leading to a better quality product and delivering more customer value.

Today the WebdriverIO team announced in their OpenJS Foundation Q&A session that the project opens an OpenCollective to allow users and companies to donate to the development of the project and support community members to run WebdriverIO workshops and other types of events. This allows everyone to be paid to work on features and bug fixes as well as help the community to pay for pizza or rental space when hosting WebdriverIO workshops or meetups.

If your company uses WebdriverIO and benefits from it, please ask your manager or your marketing team to support the project by donating to the collective. Support will allow the maintainers to dedicate more time for maintenance and new features for everyone.

As part of this we also like to announce the official WebdriverIO Swag Store: shop.webdriver.io, which will be used as an income source for the collective. All purchases from that store will directly go into our fund and to you. We start our collection with a T-shirt, a sweatshirt and a couple of accessories but will soon extend to more. So stay tuned!

We want to be transparent in the way we accept expenses to the collective. Everyone should be eligible to participate and send in expenses for development on certain features. We also want to give back to the community by allowing us to expense event expenses. The following expense types may be eligible to be reimbursed from the collective:

Event Expenses​

If you host an event that has a speaker talking about using WebdriverIO and it's features you can expense up to $100. Reimbursement requirements for event expenses include:

• You or the event account must share the project on social media (Twitter, Facebook or LinkedIn) at least 3x
• The event page must have the WebdriverIO logo and a link to the project page in your meetup description
• You must use the funds for qualified event expenses such as food, beverage, room or equipment rental.
• You must submit receipts with your reimbursement request.

Our goal with this expense policy is to help the community to run events, to promote the project and support anyone learning about WebdriverIO. We've already seen project meetups happening in New York and The Netherlands and hope to see more of this in the future.

Development Expenses​

If you have done development work on any of the repositories within the GitHub WebdriverIO organisation you may reimburse up to $1000 if the following requirements are met:

• You must have submitted qualifying pull requests that have closed at least 10 issues that were labeled with Expensable 💸
• Every additional issue closed with that label can be expensed with $100
• You must submit links to all issues you’ve closed due to your pull requests
• In order to close the ticket automatically, you must have one commit message with the Fix keyword. For example, Fix #1234 to close ticket #1234.
• Pull Requests must be merged by someone from the core team. If there are several Pull Requests, the core team member either selects the most recent one or the best one - that’s up to them to decide what is best for the project.
• You must claim an Expensable 💸 issue by commenting to the issue thread to ensure that no one else is working on the same issue.
• Anyone contributing to WebdriverIO is eligible to expense their work if the implemented features or bug fixes are not objectives of a commercial job.

Please note that contributing on behalf of a company does not make you eligible to also get reimbursed by the collective. If a company pays you to work on WebdriverIO we see this already as a contribution to the project by the company.

We understand that $1000 for fixing 10 bugs or implementing 10 features may seem arbitrary because some work items will take longer than others. However by requiring someone to finish 10 work items we hope that this uncertainty of effort and time will balance out. We also understand that this will not reimburse someone by their market value and it certainly won't allow someone to work full time on WebdriverIO. The goal of the collective is to reimburse people that usually would contribute in their free time in a very transparent and open way.

Travel Expenses​

If you are a member of the Technical Steering Committee team you are eligible to expense flights and hotel accommodations for travel to conferences or meetups as part of a speaking engagement on WebdriverIO, not paid by the event itself or a company. You may expense up to $500. Reimbursement requirements for travel expenses include:

• You must send out a post from your main social media account (e.g. Twitter, LinkedIn or personal blog) thanking all contributors of the collective after the event took place.
• You must use the funds for qualified travel expenses such as ground or air transportation to the event and hotel accommodations.
• You must submit receipts with your reimbursement request.

Even though currently there are no in-person events happening due to the COVID-19 pandemic, we hope that we come back to times where we can reconnect with the community on a personal level. For that we would like to support the core team with their travel expenses.

Overall we hope that this expense policy is fair and inclusive enough that anyone can participate. Opening a collective might sound easy in the beginning but doesn't end up being a trivial effort when you consider transparency and fairness. We expect to make amendments to our policies once we gather more experience from the process.

Thank you to everyone who will donate money to the collective and therefore will support the project and everyone contributing to it. It really means a lot ❤️