Web Driver IO Tutorial

Web Driver IO Tutorial With Live Working Web Site And Real Examples

Last Update: 02/18/2017
(currently updating for latest versions)

(Check back often - I update the original post)

Background

I recently had an interesting challenge presented to me. I needed to introduce automated testing to a Q/A department with very little technical experience and no programming background.

This was really two (2) separate challenges. The first was to research the technologies to use to do the automated testing. The second was to train the Q/A department.

The article (blog) will only address the technologies used and what I learned in the process.

The technologies worked well but I really had to search for information and I spent many hours figuring out issues. I had a hard time finding information on the Internet about these technologies all working together.

I wanted to share this information, so I wrote this article (blog) along with working example test scripts and a test web site to run the scripts against.

All test scripts can be found on Github and the working test site is located at Web Driver IO Tutorial Test Site to test the scripts against.

I hope you find it useful. If you do, please let me know. I would like to hear from you.

Objectives

Use Technologies that:

Can test web site functionality
Can test JavaScript functionality
Can be run manually
Can be run automatically
** Have easy to learn language for non programmers **

Q/A personnel with basic knowledge of HTML and JavaScript

Uses open source software only (with exception of cloud based testing platforms)
Can test multiple OS/Browser versions and combinations

** most difficult objective by far **

Technologies

List of technologies I choose:

Mocha – test framework and runner and executes the test scripts (test runner)
Shouldjs – expressive, readable, assertion library (test if something is true)
Webdriverio – browser control bindings (JS programming language bindings)
Selenium – browser abstraction and running factory (starts and communicates with browser)
Grunt - javascript task runner (used for cloud based platform testing - SauceLabs)
Grunt-WebDriverIO - grunt plugin for mocha/web driver IO (used for cloud based platform testing - Saucelabs)
wdio – wdio test runner

wdio-spec-reporter – wdio spec reporter
wdio-mocha-framework – mocha framework for wdio

Browser/Mobile drivers + browsers

Firefox (Browser and driver)
Chrome (Browser and driver)
Internet Explorer (Browser and driver)
Safari (Browser and driver plug-in)
Opera (Browser)

Saucelabs - cloud based testing platform

(Not Web Driver IO related but very good information)

supertest - test REST APIs
json-server - mock REST API server
mongodb + promise-mongo - test database driven testing
node-xlsx + excel parser & builder

Software Installation

To get started you need to have Node JS, WebDriver IO, Mocha, Should, Selenium stand alone server, grunt and grunt-webdriver plug-in installed.

Windows 7 - Manual (Global)

Here are manual instructions for global installation on Windows 7:

Note: I installed all software below using the npm global option (-g). This is normally not recommended but for this installation I needed to install globally since it would be used across multiple projects.

(I'm a Mac/Linux user but I had to install everything on Windows 7 machines, this is why I have included it for your reference. The procedure for installing on a Mac/Linux is similar. Please consult with online references for more information.)

From a browser:

Install Node which includes NPM (Node Package Manager)
go to https://nodejs.org/

Click install
Save and run file

Set the path and variable (NODE_PATH)

Go to Control Panel->System and Security->System

Advanced System Settings

Environment Setting (User variables)

Add to PATH

C:\Users\{USERNAME}\AppData\Roaming\npm;

Add the NODE_PATH (System variables)

C:\Users\{USERNAME}\AppData\Roaming\npm\node_modules

Open command prompt (cmd):

(local user administrator)

Install "web driver IO"

npm install webdriverio -g

This will install web driver IO globally including the wdio test runner on your machine

Install “mocha” test runner software

npm install mocha -g

This will install mocha globally on your machine

Install “should” assertion library

npm install should -g

This will install “should” globally on your machine

Install "grunt" task runner

npm install grunt -g

Install "grunt-webdriver" grunt plugin for webdriver

npm install grunt -g

Install Selenium Stand Alone Server

http://www.seleniumhq.org/download/
Download jar file. Save/move into the “selenium” directory.

Install browsers and browser drivers to test with:

From cmd prompt:
Create “selenium” directory

C:\Users\{USERNAME}\selenium
Commands:

cd C:\Users\{USERNAME}

mkdir selenium

Firefox

Install Firefox browser, if not already installed.

The path has to be set to start the Firefox browser from command prompt (cmd).

Control Panel->System and Security->System

Advanced System Settings

Environment Settings

Add (append use semi-colon) to Path Variable

C:\Program Files (x86)\Mozilla Firefox

Firefox driver (GeckoDriver)

Go to https://github.com/mozilla/geckodriver/releases
Download latest driver

Chrome

Install Chrome browser, if not already installed.

The path MAY set to start Chrome from command prompt (cmd)

Test first: chrome.exe from command prompt (cmd)
If Chrome doesn't start then:
Control Panel->System and Security->System

Advanced System Settings

Environment Settings

Add (append use semi-colon) to Path Variable

C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

A special web driver is needed for chrome.

chromium.org

Download and unzip 64 bit driver into the “selenium” directory.

Internet Explorer (for Windows only - will not work on other platforms)

A special web driver is needed for Internet Explorer

http://www.seleniumhq.org/download/

Download and unzip 64 bit driver into the “selenium” directory.

Mac / Linux

From a browser:

Install Node which includes NPM (Node Package Manager)
go to https://nodejs.org/

Click download node
Save and run file

Node.js was installed at: /usr/local/bin/node
npm was installed at: /usr/local/bin/npm
Make sure that /usr/local/bin is in your $PATH

From a shell prompt:

Verify node version

$ node -v

Create selenium directory:

$ mkdir selenium

Install Selenium Stand Alone Server:

Go to http://www.seleniumhq.org/download/
Download latest jar file.
Save/move into the “selenium” directory.
Add selenium directory to PATH

Firefox browser (my default browser for scripts is firefox)

Install Firefox browser, if not already installed.

Firefox driver (GeckoDriver)

Go to https://github.com/mozilla/geckodriver/releases
Download latest driver
untar

$ tar -xvf <drivername>

Save/move to directory in PATH or to selenium directory

Install software

$ git clone https://github.com/onewithhammer/WebDriverIOTutorial.git
$ cd WebDriverIOTutorial
$ npm install OR $ sudo npm install

Install "web driver IO"

npm install webdriverio -g OR sudo npm install webdriverio -g

This will install web driver IO globally including the wdio test runner on your machine
You may have to install as administrator

Install “mocha” test runner software

npm install mocha -g OR sudo npm install mocha -g

This will install mocha globally on your machine
You may have to install as administrator

Basic Test Script

Let's start with some basics.

Here is a simple mocha script that will open a web site and verify the title.

// tutorial1.js
//
// This is a simple test script to open a website and
// validate title.
//

// required libraries
var webdriverio = require('webdriverio'),
  should = require('should');

// a test script block or suite
describe('Title Test for Web Driver IO - Tutorial Test Page Website', function() {

  // set timeout to 10 seconds
 this.timeout(10000);
  var driver = {};

  // hook to run before tests
  before( function () {
    // load the driver for browser
    driver = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
    return driver.init();
  });

  // a test spec - "specification"
  it('should be load correct page and title', function () {
    // load page, then call function()
    return driver
      .url('http://www.tlkeith.com/WebDriverIOTutorialTest.html')
      // get title, then pass title to function()
      .getTitle().then( function (title) {
        // verify title
        (title).should.be.equal("Web Driver IO - Tutorial Test Page");
        // uncomment for console debug
        // console.log('Current Page Title: ' + title);
      });
  });

  // a "hook" to run after all tests in this block
 after(function() {
    return driver.end();
  });
});

Observations:

You should first notice the test script is written in JAVASCRIPT (ends in .js extension).
The basic structure is almost identical for all test scripts.

Header Comments (//)
Required Libraries
Set Options (optional)
Hook: Load Browser Driver
Test Suite (describe)
Test Specs (can be many Specs in a Suite)
Hook: Clean up

The test suite begins with a describe function which takes two parameters:

String - Description of test suite

“Check page for proper verbiage”
“Verify radio button operations”

function - block of code to execute

describe(‘Description of test suite', function() { });

The test suite will contain 1 or more test spec (specification)
Specs begin with it function which takes two parameters:

String - Description of test specification

“Should be correct link text and link URL"
“Should contain correct verbiage (copy deck)

function - block of code to execute

it(‘Description of test specification', function() { });

A spec contains one or more expectations that test the state of the code
These are called assertions

The “should” library provides the assertions

In almost all cases, you will need to locate one or more elements using a selector then perform some operation on the element(s)

Examples:

Find text on a page and verify the text
Populate form fields with data and submit
Verify CSS properties of an element

Let's take a closer look at the example with comments.

Load the required libraries: web driver IO and should.

// required libraries
var webdriverio = require('webdriverio'),
  should = require('should');

Define the test suite. This suite it is called: "Title Test for Web Driver IO - Tutorial Test Page Website"

// a test script block or suite
describe('Title Test for Web Driver IO - Tutorial Test Page Website', function() {
...
});

Set the timeout to 10 seconds so the script doesn't timeout when loading the page.

// set timeout to 10 seconds
  this.timeout(10000);

Hook to load the browser driver before running the specifications "specs". The Firefox driver is loaded in this example.

// hook to run before tests
before( function () {
  // load the driver for browser
  driver = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
  return driver.init();
});

Define the test specification.

// a test spec - "specification"
it('should be load correct page and title', function () {
  ...
});

Load the website page.

.url('http://www.tlkeith.com/WebDriverIOTutorialTest.html')

Get title, then pass title to function()

.getTitle().then( function (title) {
  ...
});

Verify the title using the should assertion library.

(title).should.be.equal("Web Driver IO - Tutorial Test Page");

Hook to quit and cleanup the driver when finished.

// a "hook" to run after all tests in this block
after(function() {
  return driver.end();
});

Run the Test Script

Now let's see what the test script does when it is ran.

First start the Selenium Stand Alone Server:

(3.0.1 is the current version at the time of writing this blog. Your version may vary.

For Windows use command line (cmd):

java -jar <selenium-server-standalone-X.XX.X.jar>
# java -jar selenium-server-standalone-3.0.1.jar

For Mac or Linux, open terminal:

java -jar <selenium-server-standalone-X.XX.X.jar>
$ java -jar selenium-server-standalone-3.0.1.jar

See screenshot

Next run the test script:

For Windows use command line (cmd):

mocha <test script file name>
# mocha tutorial1.js

For Mac or Linux, open terminal:

mocha <test script file name>
$ mocha tutorial.js

See screenshot

So what happened?

Mocha invokes the script "tutorial1.js". The driver started the browser (Firefox), loaded the page and verified the title.

Example Web Site

All the examples are run against this site.

The example web site is located at: Web Driver IO Tutorial Test Page

All test scripts can be downloaded from my Github project.

Specific Examples

All code is available on my Github:

Web Driver IO Tutorial on Github

Verify Link and Link Text in an unordered list - "linkTextURL1.js"

The unordered list has an id="mylist" and the link (anchor) is the 4th list item.
The URL should be "http://tlkeith.com/contact.html"

HTML CODE:

 // Verify Contact Us link text
  it('should contain Contact Us link text', function () {
    return driver
      .getText("//ul[@id='mylist']/li[4]/a").then(function (link) {
        console.log('Link found: ' + link);
        (link).should.equal("Contact Us");
      });
  });

  // Verify Contact Us URL
  it('should contain Contact Us URL', function () {
    return driver
      .getAttribute("//ul[@id='mylist']/li[4]/a", "href").then(function (link) {
        (link).should.equal("http://tlkeith.com/contact.html");
        console.log('URL found: ' + link);
      });
  });

Verify Copyright Text - "copyright1.js"

The copyright is in the footer
This example shows 2 different ways to locate the copyright text:

by the id="copyright" as the element selector
by using xpath as the element selector

HTML CODE

// Verify Copyright text using id as element selector
  it('should contain Copyright text', function () {
    return driver
      .getText("#copyright").then(function (link) {
        console.log('Copyright found: ' + link);
        (link).should.equal("Tony Keith - tlkeith.com @ 2015-2017 - All rights reserved.");
      });
  });

  // Verify Copyright text using xpath as element selector
  it('should contain Copyright text', function () {
    return driver
      .getText("//footer/center/p").then(function (link) {
        console.log('Copyright found: ' + link);
        (link).should.equal("Tony Keith - tlkeith.com @ 2015-2017 - All rights reserved.");
      });
  });

Populate Form Fields and Submit - "formFillSubmit1.js"

Fill in the first name, last name and submit, then wait for results.
This example shows 3 methods of filling the first name input field:

by id
by xpath from input
by xpath from form->input

Also shows how to clear an input field

HTML CODE

  // Set the first name using id to: Tony
  it('should set first name to Tony', function () {
    return driver.setValue("#fname", "Tony")
      .getValue("#fname").then( function (e) {
        (e).should.be.equal("Tony");
        console.log("First Name: " + e);
      });
  });

  // Clear the first name using id
  it('should clear first name', function () {
    return driver.clearElement("#fname")
      .getValue("#fname").then( function (e) {
        (e).should.be.equal("");
        console.log("First Name: " + e);
      });
  });

  // Set the first name using xpath from input to: Tony
  it('should set first name to Tony', function () {
    return driver.setValue("//input[@name='fname']", "Tony")
      .getValue("//input[@name='fname']").then( function (e) {
        (e).should.be.equal("Tony");
        console.log("First Name: " + e);
      });
  });

  // Clear the first name using xpath from input
  it('should clear first name', function () {
    return driver.clearElement("//input[@name='fname']")
      .getValue("//input[@name='fname']").then( function (e) {
        (e).should.be.equal("");
        console.log("First Name: " + e);
      });
  });

  // Set the first name using xpath from form to: Tony
  it('should set first name to Tony', function () {
    return driver.setValue("//form[@id='search-form']/input[1]", "Tony")
      .getValue("//form[@id='search-form']/input[1]").then( function (e) {
        (e).should.be.equal("Tony");
        console.log("First Name: " + e);
      });
  });

  // Set the last name using id to: Keith
  it('should set last name to Keith', function () {
    return driver.setValue("#lname", "Keith")
      .getValue("#lname").then( function (e) {
        (e).should.be.equal("Keith");
        console.log("Last Name: " + e);
      });
  });

  // Submit form and wait for search results
  it('should submit form and wait for results', function () {
    return driver.submitForm("#search-form").then( function(e) {
      console.log('Submit Search Form');
      })
      .waitForVisible("#search-results", 10000).then(function (e) {
        console.log('Search Results Found');
      });
  });

Click Show/Hide Button and Verify Text - "showHideVerify1.js"

The text is in a show/hide element. The button controls the state.

This example shows:

Click the button to expand
Wait for the element to be visible (expanded)
Verify text

HTML CODE

  // click "More Info" button and verify text in expanded element
  it('should click more info button and verify text', function () {
    return driver
      .click("#moreinfo").then (function () {
        console.log('Clicked More Info button');
      })
      .waitForVisible("#collapseExample", 5000)
      .getText("//div[@id='collapseExample']/div").then (function (e) {
        console.log('Text: ' + e);
        (e).should.be.equal("All things good go here!");
      });
  });

Validate Form Field Errors - "formFieldValidation.js"

Use test scripts to verify correct error messages are produced.

This example shows:

Verify the error text messages and verify location (unordered list position).

HTML CODE

 
it('should contain 5 errors: first/last/address/city/state', function () {
    return driver
     .getText("//ul[@class='alert alert-danger']/li[1]").then(function (e) {
        console.log('Error found: ' + e);
        (e).should.be.equal('Please enter first name');
      })
      .getText("//ul[@class='alert alert-danger']/li[2]").then(function (e) {
        console.log('Error found: ' + e);
        (e).should.be.equal('Please enter last name');
      })
      .getText("//ul[@class='alert alert-danger']/li[3]").then(function (e) {
        console.log('Error found: ' + e);
        (e).should.be.equal('Please enter address');
      })
      .getText("//ul[@class='alert alert-danger']/li[4]").then(function (e) {
        console.log('Error found: ' + e);
        (e).should.be.equal('Please enter city');
      })
      .getText("//ul[@class='alert alert-danger']/li[5]").then(function (e) {
        console.log('Error found: ' + e);
        (e).should.be.equal('Please enter state');
      });
  });

Looping Data to Validate URL Link/Text/Page - "loopDataExample1.js"

This example shows:

Use an array of JSON data to store the link and name, then iterate

Verify each URL text and link
Click link and load page

HTML CODE


// Link data - link and text
var linkArray = [
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/tutorial1.js", "name" : "tutorial1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/linkTextURL1.js", "name" : "linkTextURL1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/copyright1.js", "name" : "copyright1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/formFillSubmit1.js", "name" : "formFillSubmit1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/showHideVerify1.js", "name" : "showHideVerify1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/dynamicBrowser.js", "name" : "dynamicBrowser.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/callbackPromise.js", "name" : "callbackPromise.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/debugExample1.js", "name" : "debugExample1.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/formFieldValidation.js", "name" : "formFieldValidation.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/common/commonLib.js", "name" : "commonLib.js"},
{"link" : "https://github.com/onewithhammer/WebDriverIOTutorial/blob/master/dataLoopExample1.js", "name" : "dataLoopExample1.js"}
];
...
 // loop through each linkArray 
  linkArray.forEach(function(d) {
    it('should contain text/link then goto page - ' + d.name, function() {
      return driver
      // make sure you are on the starting page
      .url('http://www.tlkeith.com/WebDriverIOTutorialTest.html')
      .getTitle().then( function (title) {
        // verify title
        (title).should.be.equal("Web Driver IO - Tutorial Test Page");
      })
      // find the URL
      .getAttribute('a=' + d.name, "href").then(function (link) {
        (link).should.equal(d.link);
        console.log('URL found: ' + d.link);
      })
      // go to URL page and verify it exists
      .click('a=' + d.name)
      .waitForVisible("#js-repo-pjax-container", 10000).then(function () {
        console.log('Github Page Found');
      });
    });
  });

Looping Static Data to Populate Form Fields - "loopDataExample2.js"

This example shows:

Use an array of static JSON data objects (first/last name) hard coded in script

Loop through the array data to populate form fields, then submit the form
Wait for results page
Verify first / last name on the results page

HTML CODE


// data array - firstName and lastName
var dataArray = [
{"firstName" : "Tony", "lastName" : "Keith"},
{"firstName" : "John", "lastName" : "Doe"},
{"firstName" : "Jane", "lastName" : "Doe"},
{"firstName" : "Don", "lastName" : "Johnson"}
];

...

  // loop through each dataArray 
  dataArray.forEach(function(d) {
    it('should populate fields, sumbit page', function() {
      return driver
      // make sure you are on the starting page
      .url('http://www.tlkeith.com/WebDriverIOTutorialTest.html')
      .getTitle().then( function (title) {
        // verify title
        (title).should.be.equal("Web Driver IO - Tutorial Test Page");
      })
      .setValue("#fname", d.firstName)
      .getValue("#fname").then( function (e) {
        (e).should.be.equal(d.firstName);
        console.log("First Name: " + e);
      })
      .setValue("#lname", d.lastName)
      .getValue("#lname").then( function (e) {
        (e).should.be.equal(d.lastName);
        console.log("Last Name: " + e);
      })
      .submitForm("#search-form").then( function() {
        console.log('Submit Search Form');
      })
      .waitForVisible("#search-results", 10000).then(function () {
        console.log('Result Page Found');
      })
      .getText("//h1").then(function (link) {
        console.log('Text found: ' + link);
        (link).should.equal("Welcome " + d.firstName + " " + d.lastName + ".");
      });
    });
  });

Looping Dynamic Data to Populate Form Fields - "exelDataExample.js"

This example shows:

Use an array of dynamically loaded JSON data objects (first/last name) from an excel spreadsheet (xlsx). Column A contains first name and Column B contain last name.

Loop through the array data to populate form fields, then submit the form
Wait for results page
Verify first / last name on the results page

Notes about this example:

Uses package node-xlsx to read data from Excel spreadsheet
Uses promises (Q) to:

add a wrapper to convert async xlsx.parse() to sync function
once data is returned, loop through the array of data one record (row) at a time.

// Data from spreadsheet - array or arrays

[{"name":"Sheet1",
"data":[
["Tony","Keith"],
["John","Doe"],
["Jane","Doe"],
["John","Smith"],
["Jane","Smith"],
["Don","Johnson"]
]}]

var webdriverio = require('webdriverio');
var should = require('should');
var xlsx = require('node-xlsx');
var Q = require('q');

// loopTest()
var loopTest = function (driver, fname, lname) {
    return driver
      .url('http://www.tlkeith.com/WebDriverIOTutorialTest.html')
      .getTitle().then( function (title) {
        (title).should.be.equal("Web Driver IO - Tutorial Test Page");
        //console.log('Current Page Title: ' + title);
      })
      .setValue("#fname", fname)
      .getValue("#fname").then( function (e) {
        (e).should.be.equal(fname);
        console.log("First Name: " + e);
      })
      .setValue("#lname", lname)
      .getValue("#lname").then( function (e) {
        (e).should.be.equal(lname);
        console.log("Last Name: " + e);
      })
      .submitForm("#search-form").then( function() {
        console.log('Submit Search Form');
      })
      .waitForVisible("#search-results", 10000).then(function () {
        console.log('Result Page Found');
      })
      .getText("//h1").then(function (link) {
        console.log('Text found: ' + link);
        (link).should.equal("Welcome " + fname + " " + lname + ".");
      });
};

// getExcelData()
var getExcelData = function(fname) {
  var deferred = Q.defer();
  // turn into async call
  var xlsObject = xlsx.parse(fname);
  deferred.resolve(xlsObject);
  return deferred.promise;
};

describe('Loop Test with Excel Data for Web Driver IO - Tutorial Test Page Website', function() {

...

  it('should process data records - sequentially', function() {
    var loop = Q();
    return getExcelData('testData1.xlsx').then(function(data)  {
      console.log('Records: ' + data[0].data.length);
      data[0].data.forEach(function(d) {
        loop = loop.then(function() {
          // execute the next function after the previous has resolved successfully
          console.log('First: ' + d[0] + ' Last: ' + d[1]);
          //  Read the row data (Column A or d[0] = first name, Column B or d[1] = last name)
          return loopTest(driver, d[0], d[1]);
        });
      });
      // return last so mocha knows all records are finished.
      return loop;
    });
  });

...

});

Validate CSS Properties - "cssValidation1.js"

This example shows how to:

Validate the following CSS properties:

color
padding (top, bottom, right, left)
background color

HTML CODE


  it('should contain correct color of error text', function () {
    return driver
     .getCssProperty("//ul[@class='alert alert-danger']/li[1]", "color").then(function (result) {
        console.log('Color found: ' + result.parsed.hex + " or " + result.value);
        (result.parsed.hex).should.be.equal('#a94442');
      });
  });

  it('should contain correct padding in table cell', function () {
    return driver
      // padding: top right botton left
      .getCssProperty("//table[@id='filelist']/thead/tr[1]/td[1]", "padding-top").then(function (result) {
        console.log('padding-top found: ' + result.value);
        (result.value).should.be.equal('10px');
      })
      .getCssProperty("//table[@id='filelist']/thead/tr[1]/td[1]", "padding-bottom").then(function (result) {
        console.log('padding-bottom found: ' + result.value);
        (result.value).should.be.equal('10px');
      })
      .getCssProperty("//table[@id='filelist']/thead/tr[1]/td[1]", "padding-right").then(function (result) {
        console.log('padding-right found: ' + result.value);
        (result.value).should.be.equal('5px');
      })
      .getCssProperty("//table[@id='filelist']/thead/tr[1]/td[1]", "padding-left").then(function (result) {
        console.log('padding-left found: ' + result.value);
        (result.value).should.be.equal('5px');
      });
  });

 it('should contain correct background color in table header', function () {
    return driver
     .getCssProperty("//table[@id='filelist']/thead", "background-color").then(function (result) {
        console.log('background color found: ' + result.parsed.hex);
        (result.parsed.hex).should.be.equal('#eeeeee');
      });
  });

Architecture

This section will show how do the technologies work together for local setup, grid setup and cloud based testing platform setup.

Note: Selenium can run in 3 different modes: standalone server, hub or node.

Local Selenium Setup

All software runs on your local computer

Node runs Mocha framework and runner of test script.
Should is the assertion library.
Web Driver IO communicates with Selenium Server using JSON Wire Protocol.
Selenium Server invokes local browser using a driver to test the web application.

Selenium Grid Setup

Software run on your local computer and network computers.

Node runs Mocha test framework and runner of test script.
Should is the assertion library.
Web Driver IO communicates with Selenium Hub using JSON Wire Protocol.
Selenium Hub routes requests to Selenium Nodes with different OS/Browsers combinations.

Cloud Based Testing Platform Setup

Software run on your local computer and cloud based testing platform.

Node runs Grunt and Grunt-Webdriver plug-in runs Mocha the test framework and test script.
Should is the assertion library.
Web Driver IO communicates with cloud based testing platform using JSON Wire Protocol. (saucelabs, browserstack, …)
Cloud based testing platform will automatically setup the correct OS/Browser combinations to test your application on.

Tips and Tricks:

Debugging

Use Firebug debugger in the Firefox browser

Use Firebug to inspect elements

Turn on logging at the driver level for more debug and to create logs.

Set logLevel to 'verbose'
Set logOutput to directory name ('logs')

driver = webdriverio.remote(loglevel: 'verbose', logOutput: 'logs' , {desiredCapabilities: {browserName: 'firefox'} });

Use console.log(), debug(), getText() to debug.

console.log() - Use to display information to determine state.
debug() - Use pause browser/script until enter is pressed on command line.
getText() - Use to verify you are interacting with the correct element.

Especially helpful with xpath expressions.

  // Click on the Item 3 from list
  it('should click on Item 3 from list', function () {
    // use getText() to verify the xpath is correct for the element
    return driver
      .getText("//ul[@id='mylist']/li[3]/div/div/a").then(function (link) {
        // use console.log() to output information
        console.log('Link found: ' + link);
        (link).should.equal("Item 3");
      })
      // use debug() to stop action to see what is happening on the browser
      .debug()
      .click("//ul[@id='mylist']/li[3]/div/div/a").then (function () {
        console.log('Link clicked');
      })
      // wait for google search form to appear
      .waitForVisible("#tsf", 20000).then(function (e) {
        console.log('Search Results Found');
      });
  });

Use Environment Variable to Change the Browser Dynamically

Use environment variable SELENIUM_BROWSER to invoke a different browser without modifying the test script each time.

$ env SELENIUM_BROWSER=chrome

Requires a slight coding change to support.

Code Changes:

  // load the driver for browser
    driver = webdriverio.remote({ desiredCapabilities: 
      {browserName: process.env.SELENIUM_BROWSER || 'chrome'} });

Supported Browsers:

IE 8+ (Windows Only)

SELENIUM_BROWSER=ie mocha <test script file name>

Firefox 10+ (Windows/Max/Linux)

SELENIUM_BROWSER=firefox mocha <test script file name>

Chrome 12+ (Windows/Max/Linux)

SELENIUM_BROWSER=chrome mocha <test script file name>

Opera 12+

SELENIUM_BROWSER=opera mocha <test script file name>

Safari

SELENIUM_BROWSER=safari mocha <test script file name>

Testing:

For Windows use git bash shell:

SELENIUM_BROWSER=chrome mocha <test script file name>
$ SELENIUM_BROWSER=chrome mocha dynamicBrowser.js

For Mac or Linux, open terminal:

SELENIUM_BROWSER=chrome mocha <test script file name>
$ SELENIUM_BROWSER=chrome mocha dynamicBrowser.js

Responsive Testing

Determine breakpoints based on project or framework (ie bootstrap).
Define environment variables for each breakpoint:

DESKTOP - 1200 px
TABLET - 992 px
MOBILE - 768 px

Develop a reusable command to read the environment variable and set the browser size. See example below.
Call the reusable command in your test script.

  // reusable code - library
  // code snippet
  if(bp == "DESKTOP") {
    obj.width = 1200;
    obj.height = 600;
    obj.name = bp;
  }
  else if(bp == "TABLET") {
    obj.width = 992;
    obj.height = 600;
    obj.name = bp;
  }
  else if(bp == "MOBILE") {
    obj.width = 768;
    obj.height = 400;
    obj.name = bp;
  }

  // Test script
  before( function(done) {
    winsize = common.getWindowSizeParams();
    ...
    driver.addCommand('setWindowSize', common.setWindowSize.bind(driver));
  }

  // set the window size
  it('should set window size', function (done) {
    // only the width matters
    driver.setWindowSize(winsize.width, winsize.height, function () {}).call(done);
  });

Reusable Commands (Custom Commands)

Web Driver IO is easily extendable.

I like to put all reusable commands into a library. (Maybe this is old school but it works!)

See common/commonLib.js for all reusable commands

//
//  verifyLastNameCheckError()
//
//    Description:
//      Verifies the last name form validation error message
//
//    Input:
//      number - index of error (1-5)
//    Output:
//      none
//
var verifyLastNameCheckError = function () {
    var idx = arguments[0];
    this
        .getText("//ul[@class='alert alert-danger']/li[" + idx + "]").then( function(e) {
            console.log('Error found: ' + e);
            (e).should.be.equal('Please enter last name');
        });
};
// export the function
module.exports.verifyLastNameCheckError = verifyLastNameCheckError;



// see FormFieldValidation.js for complete example

// Here are the specific changes needed to call a reusable function

  // require the reusable command - CommonLib
  common = require('./Common/CommonLib');
  ...

  // bind the commands
  driver.addCommand('verifyFirstNameError', common.verifyFirstNameCheckError.bind(driver));
  driver.addCommand('verifyLastNameError', common.verifyLastNameCheckError.bind(driver));

  it('should contain 2 errors: first/last name', function () {
    // call the reusable function
    driver
      .verifyFirstNameError(1);
      .verifyLastNameError(2);
  });

Project File/Directory Structure

Here is typical project structure:

"Project" - main project directory

README.md - readme for global project
"Common" - directory for global functions common to all projects

common-lib.js - global function library
README.md - readme for global functions

"Product1" - directory for product 1

test-script1.js
test-script2.js
"Common" - directory for local functions to project 1

prod1-lib.js - local function library for project 1
README.md - readme for local functions to project 1

"Product2" - directory for product 2

test-script1.js
test-script2.js
"Common" - directory for local functions to project 2

prod2-lib.js - local function library for project 2
README.md - readme for local functions to project 2

Break test scripts into multiple files:

Here is a sample of using multiple files:

Sanity Check - basic test script to verify everything is working
Static Element and Text Validation - verify all elements and text
Form/Page Error Validation - error validation
Search Results - test dynamic content

Callbacks VS. Promises

Version 3 of Web Driver IO supports both callbacks and promises. Promises are the preferred method since it reduces the error handling code. Please see below the same example written using callbacks and promises.

Callbacks

  // Set/verify first/last name using Callbacks
  it('should set/verify first/last name using Callbacks', function (done) {
    driver.setValue("#fname", "Tony", function (e) {
      driver.getValue("#fname", function (err, e) {
        (e).should.be.equal("Tony");
        console.log("First Name: " + e);

        driver.setValue("#lname", "Keith", function (e) {
          driver.getValue("#lname", function (err, e) {
            (e).should.be.equal("Keith");
            console.log("Last Name: " + e);
            done();
          });
        });
      });
    });
  });

Promises

  // Set/verify first/last name using Promises
  it('should set/verify first/last name using Promises', function () {
    return driver.setValue("#fname", "Tony")
      .getValue("#fname").then( function (e) {
        (e).should.be.equal("Tony");
        console.log("First Name: " + e);
      })
      .setValue("#lname", "Keith")
      .getValue("#lname").then( function (e) {
        (e).should.be.equal("Keith");
        console.log("Last Name: " + e);
      });
  });

Detecting Grunt + Grunt-Webdriver or Mocha as the test runner

Check for the existence of the global "browser" variable. If it exists, the test runner is grunt + grunt-webdriver plugin.
This allows the same script to be run from Grunt + Grunt-Webdriver or Mocha.

This test will work if mocha is the test runner.

$ mocha <filename>

This test will also run if grunt is the test runner.

$ grunt webdriver

  
  // hook to run before tests
  before( function () {
    // check for global browser (grunt + grunt-webdriver)
    if(typeof browser === "undefined") {
      // load the driver for browser
      driver = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
      return driver.init();
    } else {
      // grunt will load the browser driver
      driver = browser;
      return;
    }
  });

  ...
  // a "hook" to run after all tests in this block
 after(function() {
    if(typeof browser === "undefined") {
      return driver.end();
    } else {
      return;
    }
  });

Detecting the test runner was initiated from Travis CL

Check for the existence of the BUILD_NUMBER environment variable. If it exists, the test runner was initiated by Travis CL.
This allows the same script to be run manually and connect to saucelabs.com on port 80, or if the script is run by Travis CL, then connect to localhost on port 4445.

If BUILD_NUMBER exists, set the host to 'localhost' and the port to 4445, else set host to 'ondemand.saucelabs.com' and the post to 80.

  Snippet of Gruntfile.js

      options: {
          host: (process.env.BUILD_NUMBER) ? 'localhost':'ondemand.saucelabs.com',
          port: (process.env.BUILD_NUMBER) ? 4445:80,
          user: process.env.SAUCE_USERNAME,
          key: process.env.SAUCE_ACCESS_KEY,
          tags: ['saucelabs'],
          name: 'This is an example test script using grunt-driver and saucelabs'
      },

Using different runners

Lets recap:

mocha is the test framework
should is the assertion library
In most of the examples, mocha is also the runner.

$ mocha <filename>

Using mocha as the runner

Run locally a single test using mocha as framework and runner:

$ mocha [test-script-filename]

Run locally single test using mocha as framework and runner:

$ mocha [test-script-filename]
$ mocha tutorial1.js

Using grunt plugin (grunt-webdriver) to invoke wdio runner

Run grunt with default config file (Gruntfile.js) using mocha as the framework. The config file will run a few test files against saucelabs with different OS/Browser combinations.

Note: Gruntfile.js calls wdio.conf-gruntfile.js

$ grunt [task-name]
$ grunt webdriver

You will need a saucelabs account in order to set the environment variables for SAUCE_USERNAME & SAUCE_ACCESS_KEY
$ export SAUCE_USERNAME=[your saucelabs username]
$ export SAUCE_ACCESS_KEY=[your saucelabs access key]
OR
$ grunt --gruntfile <config-filename> [task name]
$ grunt --gruntfile Gruntfile-dataLoopExample2.js webdrive

Run wdio as runner:

Run locally single test using mocha as framework and wdio as the runner:

$ wdio [config-filename]
$ wdio wdio-conf.tutorial1.js

$ wdio wdio-conf.dataLoopExample2.js

Run on saucelabs a single test using mocha as framework and wdio as the runner on 2 OS/browsers:

$ wdio [config-filename]

$ wdio wdio-conf-saucelabs.dataLoopExample2.js

More Resources:

Here are some additional resources for your reference:

Discussion Groups (Gitter)

Other interesting projects

Conclusion:

I spent some time researching the technologies to use. I originally started with Selenium Web Driver but switched to using Web Driver IO. Web Driver IO seemed to be easier to use and much easier to extend (at least the documentation for extending - reusable commands was better).

When I first started looking at the technologies it was hard to find good examples that were relative to anything I was trying to do. This is the reason I wanted to share this information and knowledge with you.

These technologies worked much better than I expected however there was learning curve involved. Once I understood how all the components worked together, I was able to write complicated test scripts in a very short time. The most difficult scripts were JS based components such as a date picker and modal selectors.

I have never labeled myself as a JavaScript developer nor did I every want to be JavaScript expect, but using these technologies has definitely motivated me to sharpen my JavaScript skills.

I hope this article is useful and the examples are clear and informative.

Please let me know if you have any questions or comments.

Thank you,
Tony Keith

Web Driver IO Tutorial

Wednesday, July 8, 2015

Original Post