Automated Visual Testing with Selenium and Applitools

      UI comparison across multiple resolutions with different dimensions such as Content, Layout, cross browsers is a difficult task with the help of manual testing. 

Why should visual testing be automated:
1. Lot of areas needs to be covered such as browsers, devices, screen resolutions, RWDs etc.
2. Manual visual testing is error prone and time taking.
3. In the process of “Continues Deployment”, it is mandatory that every task should be automated.

In the search of a automated tool for this purpose, many things comes in mind. Particularly, when we think about open source tools, selenium will be the first choice. But we cannot accomplish the task with selenium due to certain limitations such as desktop application support, iOS support, screenshots comparison etc.

Now in Paid tools, there is a tool named applitools which is designed for this particular purpose. The Applitools is the Applitools Eyes automatically validates the correctness of the UI layout, content and appearance on all browsers, devices and screen resolutions, and enables to automate tests that can only be done manually without it. 

Features of Applitools:

• Automated testing of all the visual aspects of your application:
• One simple test validates all the fields on a given screen.No need to write separate test for each UI element on the screen
• Easy integration with existing test automation and ALM frameworks:
• Test automation tools: Selenium, Appium, MS Coded UI, HP QTP (coming soon) and more.ALM tools: HP QC, MS TFS, IBM Rational Quality Manager, Atlassian, Rally and more.Cloud testing environments.
• Seamless testing on multiple platforms, screen-resolutions and form-factors:
• Define expected results or volatile areas for one browser, one screen resolution or one form factor and all tests would automatically be updated in all browsers, all screen resolutions and all form factors. Any change in your application should only be approved once in one specific test and all other tests would automatically be updated for all browsers, all screen resolutions and all form factors
• Visual self-explanatory logs:
• No more reading of complex text-based failure logs.
  Visual test timeline points you immediately to any failed steps.
  Smart visual test playback shows step by step the user actions and the screen displays during a test.

You can find more about applitools from https://applitools.com/

Now in this article we are going to learn about integrating applitools with selenium.

1. Download the latest version of the Eyes Java Selenium SDK from https://store.applitools.com/download/eyes_selenium_java/ and extract it to a folder of your choice. Add the extracted files to your path.
2. When running tests, make sure to use your personal API key: XFkqR77qRm9GZkvw9STuox8gBrECZt298F1101ur39cioY110

Sample code is as below:
public static void main(String[] args) throws URISyntaxException, InterruptedException {
        WebDriver driver = new FirefoxDriver();
        Eyes eyes = new Eyes();
        // This is your api key, make sure you use it in all your tests.
        eyes.setApiKey("XFkqR77qRm9GZkvw9STuox8gBrECZt298F1101ur39cioY110");
        try {
            // Start visual testing with browser viewport set to 1024x768.
            // Make sure to use the returned driver from this point on.
            driver = eyes.open(driver, "Applitools", "Test Web Page", new 
            RectangleSize(1024, 768));
            driver.get("http://applitools.com");
            // Visual validation point #1
            eyes.checkWindow("Main Page");
            driver.findElement(By.cssSelector(".features>a")).click();
            // Visual validation point #2
            eyes.checkWindow("Features page");
            // End visual testing. Validate visual correctness.
            eyes.close();
        } finally {
            // Abort test in case of an unexpected error.
            eyes.abortIfNotClosed();
            driver.quit();
        }
    }
 }
3. after running a test for the first time, it will automatically be saved as a baseline for future test runs. Now run your test again and open Applitools eyes to analyze the changes. For each step either 'Accept New' in    case the changes are expected or 'Keep Baseline' in case the changes reflect a bug.

Advantages of this integration:

• Single page validation replaces hundreds of lines of validation code and hours of manual testing.
• Automatic maintenance – approved changes are automatically propagated to other tests and execution environments.
• Cognitive vision – avoiding false positives, by ignoring differences that are invisible to the human eye.
• Automatically test your app in multiple languages.
• Easily tests complex application pages including dynamic content, size and moving elements.
• It takes the pressure off manual QA and increases the coverage, test faster & more accurately.

This is one of the effective tool to automate Visual Testing in Mobile, Web, Desktop Applications. Happy Testing :) 

Do Makeup to your Jenkins Server

   In continuous Integration Process Jenkins plays a major role in terms of providing a very good platform to perform the task. It gives us the basic requirements such as normal UI, Trend chart. However it is always useful to have comprehensive dash boards, trend charts for better tracking purpose and eye catching UI themes that makes difference. We can also change the email configurations such as Subject lines, body content, including the report into the body of the email etc. For every requirement that is mentioned above, jenkins provides us the plugins. We will tweak those plugins and add the changes that we need. 

HTML Publisher

This plugin is handy with built-in copy to master functionality. Imagine that you have some files generated through your builds that are not actual artifacts. Let’s say logs, execution reports. This plugin lets you to copy them to a master and provide an easy HTML file that will have links to these special files of yours for really quick access.

    To configure this follow the below steps:
1. Click on the Configure option for your Jenkins job.
2. In the post build portion, look for the Publish HTML Reports option and select the check box. See the screen shot below

Fill the path to the directory containing the html reports in the "HTML directory to archive" field. Specify the pages to display (default index.html); you can specify multiple comma-separated pages and each will be a tab on the report page. Finally, give a name in the Report Title field, which will be used to provide a link to the report. By default, only the most recent HTML report will be saved, but if you'd like to be able to view HTML reports for each past build, select "Keep past HTML reports."

Some time at HTML directory, you don't have to specify project name, but have to start one level below. For e.g. if your project name is "ABC" and if HTML files are at "ABC/report-output/html" then you have to specify just \test-output\html\.

3. After saving the configuration, run build once. The published HTML reports are available to view from within Jenkins with convenient links in the dashboard.

Green Balls

Historically Jenkins has used the blue color to denote successful builds. So, all status messages throughout the GUI use blue as the color of success. But if you feel that green is the way to go (Green Hornet, Green Lantern, Gumby, traffic lights, etc) then you can change your preference to green with the Green Balls plugin.

Simple Theme Plugin:
This is a plugin for Jenkins that supports custom CSS & JavaScript.
You can customize Jenkins's appearance (ex. his gentle face on the background) as below:

Navigate to Jenkins > Manage Jenkins > Configure System > Theme
Set URL of theme CSS and JS to URLs of your customized CSS and JS

Some example themes are as below:
http://daniilyar.github.io/Jenkins-themes/themes/canon-jenkins/min/styles.css
http://daniilyar.github.io/Jenkins-themes/themes/canon-jenkins/min/script.js

Wall display Plugin:

A wall display that shows job build progress in a way suitable for public wall displays. Rendering is performed using javascript based on REST API calls, so requires no page refreshes.

After installing the plugin you will get a option on the left hand sidebar.To get it display simply click on the 'Wall Display' link in the sidebar of your project.

Email Extension Plugin:
This plugin allows you to configure every aspect of email notifications. You can customize when an email is sent, who should receive it, and what the email says. 

After installing this, go to the job, which has to configured for mail trigger, Click on “configure”. Go to the postbuild actions, click on “Editable Email Notification”. This will have options like the recipent details etc. Select “HTML(text/Content)” for “Content Type” option. under the Default content text area type the below text as it is though it contains html tags.

Hi All,

 Please find the overall Results of this build as below. You can click on http://10.40.60.186:8080/view/MCS/job/LETSCRETSAutomation/HTML_Report/ for more details.

${FILE,path=”test-output/emailable-report.html”}

Bottom of the line in the above script specifies the path. Change it according to your html path. Remember here, if ur html report depends on any resources like images etc, we have configure them accordingly.

https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

Dash Board Plugin:
This plugin contributes a new view implementation that provides a dashboard / portal-like view for Jenkins.
The configuration is done in 2 parts, selecting the Jenkins jobs to include in the view, and selecting which dashboard portlets to have included in the view.  The jobs that you select are used for all the portlets to draw their information from.