Finding JavaScript errors through Webdriver

        Now a days it became very common that many of the applications are using extensive java script for the client side transactions. So, from the QA perspective it is very important to validate those Java script errors irrespective of whether those errors or warnings affect the application in near future or not. There are options like firebug, chrome developer tool, IE Developer tool, which gives the java script errors on their consoles. 

         We can automate this by using Selenium Web driver API. Below code snippet can be used to get those java script errors. Here I used one sample site which has some java script errors

     public void consoleEntries(LogEntries consoleEntries) {
         for (LogEntry logEntry : consoleEntries) {
               System.out.println("Log Level: " + logEntry.getLevel().toString());
               System.out.print(" Log Message: " + logEntry.getMessage().toString());
          }
    }


  @Test
  public void getJavaScriptErrors() throws Exception {
  String url = "http://www.softwaretestingtricks.com/";
  WebDriver webDriver = new FirefoxDriver();
  webDriver.get(url);
   LogEntries logEntries = webDriver.manage().logs().get(LogType.BROWSER);
   consoleEntries(logEntries);
  webDriver.quit();
  }


To reduce the no.of logs, we can set the log preferences. So, that we will get only desired logs. That can be done as below

        DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
LoggingPreferences loggingPreferences = new LoggingPreferences();
loggingPreferences.enable(LogType.BROWSER, Level.SEVERE);
desiredCapabilities.setCapability(CapabilityType.LOGGING_PREFS, loggingPreferences);
webDriver = new FirefoxDriver(desiredCapabilities);

Winium for automating Desktop Applications

Winium is a Automation framework for Windows Applications. It is a open source tool from 2GIS. Winium supports

• Windows Desktop (WPF, WinForms) Apps
• Windows Store or Universal Apps for Windows Phone
• Windows Phone Silverlight Apps

But in this post we are going to discuss only about Winium for Desktop application automation.

We know that selenium supports only web applications. There are multiple other tools that we discussed in our previous blogs like AutoIT, Sikuli can be used to automate Windows based applications. But winium has the below advantages comparing to AutoIT and Sikuli.

1. As discussed in our previous blogs, to use Auto IT, We have to use some windows bridge to access the methods in it. But Winium comes as Java API and also it is implemented on JSONWire protocal that is used by selenium.
2. We can also write tests in any programming language that is supported by selenium.
3. It supports wide magnitude of applications comparing to AutoIT.

Technically, Winium.Desktop is an http client. It implements JSWP protocol and uses Cruciatus to work with UI elements. Essentially, this is an implementation of WebDriver for Windows-based desktop applications. It uses regular Selenium bindings with Winium. Desktop in order to test Windows-based desktop applications.

Below are the features of Winium.

• Automates native windows desktop app(winforms, WPF, everyapp that uses MS accessibility)
• compatible with JSON wire protocal
• Full access to desktop and UI
• Protocal extensions for desktop specific elements
• We can write tests in any programming language which Selenium Web driver supports

Below are commands that are supported by Winium.

Command	Query
NewSession	POST /session
FindElement	POST /session/:sessionId/element
FindChildElement	POST /session/:sessionId/element/:id/element
ClickElement	POST /session/:sessionId/element/:id/click
SendKeysToElement	POST /session/:sessionId/element/:id/value
GetElementText	GET /session/:sessionId/element/:id/text
GetElementAttribute	GET /session/:sessionId/element/:id/attribute/:name
Quit	DELETE /session/:sessionId
ClearElement	POST /session/:sessionId/element/:id/clear
Close	DELETE /session/:sessionId/window
ElementEquals	GET /session/:sessionId/element/:id/equals/:other
ExecuteScript	POST /session/:sessionId/execute
FindChildElements	POST /session/:sessionId/element/:id/elements
FindElements	POST /session/:sessionId/elements
GetActiveElement	POST /session/:sessionId/element/active
GetElementSize	GET /session/:sessionId/element/:id/size
ImplicitlyWait	POST /session/:sessionId/timeouts/implicit_wait
IsElementDisplayed	GET /session/:sessionId/element/:id/displayed
IsElementEnabled	GET /session/:sessionId/element/:id/enabled
IsElementSelected	GET /session/:sessionId/element/:id/selected
MouseClick	POST /session/:sessionId/click
MouseDoubleClick	POST /session/:sessionId/doubleclick
MouseMoveTo	POST /session/:sessionId/moveto
Screenshot	GET /session/:sessionId/screenshot
SendKeysToActiveElement	POST /session/:sessionId/keys
Status	GET /status
SubmitElement	POST /session/:sessionId/element/:id/submit

Disadvantages:

• It uses real mouse and key board events, i.e You can't run more than one session on same machine, or use mouse while tests are running.
• If the app runs in the background and remains running there, then it will not work appropriately.
• It is sometimes difficult to get the text from component that displays the output. Then you receive the error message:  NO Text property. The reason is probably that this type of component is defined as Image.

In this post we are going to learn about writing a text into notepad by using winium. Please follow the steps given below.

Step 1:
Download the below jars
http://mvnrepository.com/artifact/com.github.2gis.winium/winium-elements-desktop/0.1.0-1
http://mvnrepository.com/artifact/com.github.2gis.winium/winium-elements-desktop/0.2.0-1

Along with these two jars you need to have selenium latest jar as well. You can get the latest selenium jar from the below path.
http://docs.seleniumhq.org/download/

Step 2:
Download Winium.Desktop.Driver.exe file from https://github.com/2gis/Winium.Desktop/releases (Latest). Extract the zip file

Step 3:
Run the Winium.Desktop.Driver.exe file (by double clicking the Winium. Desktop.Driver.exe file). So that a Winium desktop server will be started and running at port 9999.

Step 4:
Create a sample Java project and add the downloaded jars into the class path of the project and also TestNG Library. Now create a class and use the below code.

package com.src.Nainappa.test;
import java.io.IOException;
import java.net.URL;
import org.openqa.selenium.winium.DesktopOptions;
import org.openqa.selenium.winium.WiniumDriver;
import org.testng.annotations.Test;

public class NotepadTest {
@Test
public void test() throws IOException{
DesktopOptions options= new DesktopOptions();
options.setApplicationPath("C:\\WINDOWS\\system32\\notepad.exe");
try{
WiniumDriver driver=new WiniumDriver(new URL("http://localhost:9999"),options);
driver.findElementByClassName("Edit").sendKeys("This is sample test");
driver.close();
}
catch(Exception e){
System.out.println(e.getMessage());
}
}
}

Step 5:
Run this code as TestNG Method. It opens the notepad and enters the text on the note pad and also tries to close it.

If anybody is wondering how to get to know the properties of the elements, we have multiple ways.

1. Renorex offers free spy. We can download it from the below path and use it.
   http://www.ranorex.com/test-automation-tools/ranorex-spy.html

2. The other way is White tool. You can download the entire tool from the below path.
   https://github.com/TestStack/White
   Extract the Zip file and you will see a file called UISpy.exe. This can also be used for spying on the elements.

We will see automating the Windows phone applications in our next post.

The role of machine learning in Automated Software Testing...

Coming soon....

Galen Framework for RWD/Layout/Cross Browser Automation

   Automating the layout testing for websites is always a challenging. But is very important in the current industry as the Responsive Web Design has become more popular than ever in the Front End Development. Testing these responsive web application manually is a herculean task as the testing spans across multiple resolutions,devices and browsers. Huge amount of manual effort and infrastructure support is required in order to  cover the above scenarios.

      In the quest of finding out sophisticated tool I had two things in mind. First thing was, tool should be able to integratable with any Selenium based automation frameworks by considering its usage in the industry. Second thing was, tool should be Open source. I was able to find out a very good tool which can be integrated with Selenium named Applitools. But applitools is a Paid tool. I have already shared a post that explains how applitools can be used with selenium framework. You can find the post here.

http://seleniumocean.blogspot.com/2015/08/automated-visual-testing-with-selenium.html

    This time I am going to share a Open source tool which can be used for RWD Layout tests.

    Galen Framework is built on top of selenium. This checks each individual component of the page with another element.The fundamental testing concept in Galen Framework centers on checking the location and size of all page elements relative to each other. This way, you can describe the layout for any browser window’s size and you don’t have to use absolute positioning. Selenium offers functionality for getting location and dimension of element in all browsers. When it comes to testing a responsive layout it works in a following way:

1. Open a page in browser

2. Resize it to specified size

3. Test the layout according to user-defined specs

     Galen allows you to express your expectations towards website layout and then uses selenium to retrieve the information about the page elements. These expectations should be written in a special laguage called GalenSpec language. The Galen Specs language was designed to resemble natural English as closely as possible and has been implemented in a semi-Markdown way. The below link gives you  more details about Galen Spec Language.

http://galenframework.com/docs/reference-galen-spec-language-guide/

    Once your test is completed Galen provides very comprehensive reporting. Galen provides good explanations using Object Definition names in reports when a test failed or passed. You can also have a nice screenshot in a popup after clicking on the failed test.It highlights the failed object on the screenshot. It will show you what's wrong at a glance. These reports would automatically generated under "target/galen-html-reports". But these can also be saved into user specific HTML Reports.

Once you click on any report, it opens the below view with hierarchical details.

  The initial version of Galen Framework was in JavaScript. But now it is supporting Java as well. Here is the sample project with Galen Java API. 

https://github.com/galenframework/galen-sample-java-tests

You can download this project and add your own pages with the Galen Spec Language. This can also be integrated with the existing java and selenium based automation frameworks by its Maven dependency. 

Below are some other features of Galen:

1. We can use it for image comsprison as well with percentage  of tolerance

2. This can be iintegrated with saucelabs for cloud based devices

3. This can be used for internatiolization

4. Galen also tests the Colors of your website

5. Galen is very good for cross browser testing

6. This will fit into the behavior testing processes such as BDD,TDD etc.

Conclusion:

In my personal opinion, this is a very good open source tool for RWD automation. But if you are looking for only layout testing, I would use it for the main lines of a project like the global layout of a given page or when a feature really needs a particular layout. I would recommend using Galen aside your daily tests, it's just another security before going in production. This can also be used by the front end developers before they push code changes to QA. 

Cobertura War Instrumentation and Coverage in a web application

In a day to day work we typically run our tests and presumably check that we are getting the expected results.Our tests may all pass with flying colours, but if we've only tested 50% of the code, how much confidence can we have in it? So a step ahead would be to ensure how much of the application code is exercised by our tests which ensures the quality of the tests that we typically run and thus code coverage comes into picture.

What is Code Coverage?
Code Coverage is a measurement of how many lines/blocks of the application code are executed while the automated/unit tests are running. Though code coverage is a white box testing methodology(as it requires knowledge of and access to the code itself rather than simply using the interface provided), we might come across situations where we need to analyse/collect code coverage metrics for our functional test cases.This post briefs the need of code coverage,coverage analysis,instrumentation for functional test cases with an example using Cobertura.

How is Code Coverage achieved?
Code coverage is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated/unit tests against the instrumented product. A good tool will give you not only the percentage of the code that is executed, but also will allow you to drill into the data and see exactly which lines of code were executed during particular test. There are many coverage tools in market for Java Code Coverage -Cobertura,Clover,Emma,Jcoverage.In this post we will generate coverage metrics using cobertura -one of the most widely used java coverage tool as it has edge over other tools.

Features of Cobertura:

• Open source Java tool.
• Can be executed from simple ant build script.
• Instruments Java bytecode after it has been compiled.
• Can generate reports in HTML/XML by class name, percent of lines covered, percent of branches covered, etc.
• Shows the percentage of lines and branches covered for each class, each package, and for the overall project.

What is instrumentation?
Instrumentation is all about manipulating the application code by injecting reporting code into strategic positions. In fact, the art of instrumentation falls in either of the two: class instrumentation and source instrumentation. Not surprisingly, the difference is that class instrumentation injects the reporting code directly into compiled .class files while source instrumentation creates an intermediary version of the sources which are then compiled into the final, source-instrumented .class files. Most of the code coverage tools will follow either of these instrumentation techniques.

For this post we are using a sample war file from (https://tomcat.apache.org/tomcat-6.0-doc/appdev/sample/)for instrumentation wherein for real time scenario,this should be replaced with your application war file.

Sample Cobertura War Instrumentation and Coverage in a web application

PreRequisite:

• Tomcat 7 and above
• JDK 1.7 and above
• Ant 1.8 and above
• Cobertura 1.9 and above
• A sample war file for instrumentation/use your application war file

Pre-coverage steps:

• Extract classes from your war.
• Create an ant script for instrumenting the classes.
• Add required libs/dependencies for instrumentation (Sample attached- Refer build.xml and build.properties).

Coverage steps:

• Perform Cobertura instrumentation using your build target as below.
• Now we have instrumented classes generated as below ,Replace the generated files in the war (replace with the existing classes) and cobertura.ser file generated.
• Put your updated war in the tomcat webapps.
• Put your cobertura.ser file in the tomcat bin .
• Add cobertura.jar in the tomcat lib .
• Start your tomcat.
• Hit the required test cases (run ur web application flows either manually or through your automation scripts)
(for this sample war -we are hitting http://localhost:8080/sample home page -this application has Hello page-hello class alone so we are executing a test to hit the hello page)
• Now put the updated cobertura.ser in the project ant home folder

Post-coverage steps:

• Run coverage targets as below.
• Now we have coverage for the test cases that we have run!

Automated Visual Testing with Selenium and Applitools

      UI comparison across multiple resolutions with different dimensions such as Content, Layout, cross browsers is a difficult task with the help of manual testing. 

Why should visual testing be automated:
1. Lot of areas needs to be covered such as browsers, devices, screen resolutions, RWDs etc.
2. Manual visual testing is error prone and time taking.
3. In the process of “Continues Deployment”, it is mandatory that every task should be automated.

In the search of a automated tool for this purpose, many things comes in mind. Particularly, when we think about open source tools, selenium will be the first choice. But we cannot accomplish the task with selenium due to certain limitations such as desktop application support, iOS support, screenshots comparison etc.

Now in Paid tools, there is a tool named applitools which is designed for this particular purpose. The Applitools is the Applitools Eyes automatically validates the correctness of the UI layout, content and appearance on all browsers, devices and screen resolutions, and enables to automate tests that can only be done manually without it. 

Features of Applitools:

• Automated testing of all the visual aspects of your application:
• One simple test validates all the fields on a given screen.No need to write separate test for each UI element on the screen
• Easy integration with existing test automation and ALM frameworks:
• Test automation tools: Selenium, Appium, MS Coded UI, HP QTP (coming soon) and more.ALM tools: HP QC, MS TFS, IBM Rational Quality Manager, Atlassian, Rally and more.Cloud testing environments.
• Seamless testing on multiple platforms, screen-resolutions and form-factors:
• Define expected results or volatile areas for one browser, one screen resolution or one form factor and all tests would automatically be updated in all browsers, all screen resolutions and all form factors. Any change in your application should only be approved once in one specific test and all other tests would automatically be updated for all browsers, all screen resolutions and all form factors
• Visual self-explanatory logs:
• No more reading of complex text-based failure logs.
  Visual test timeline points you immediately to any failed steps.
  Smart visual test playback shows step by step the user actions and the screen displays during a test.

You can find more about applitools from https://applitools.com/

Now in this article we are going to learn about integrating applitools with selenium.

1. Download the latest version of the Eyes Java Selenium SDK from https://store.applitools.com/download/eyes_selenium_java/ and extract it to a folder of your choice. Add the extracted files to your path.
2. When running tests, make sure to use your personal API key: XFkqR77qRm9GZkvw9STuox8gBrECZt298F1101ur39cioY110

Sample code is as below:
public static void main(String[] args) throws URISyntaxException, InterruptedException {
        WebDriver driver = new FirefoxDriver();
        Eyes eyes = new Eyes();
        // This is your api key, make sure you use it in all your tests.
        eyes.setApiKey("XFkqR77qRm9GZkvw9STuox8gBrECZt298F1101ur39cioY110");
        try {
            // Start visual testing with browser viewport set to 1024x768.
            // Make sure to use the returned driver from this point on.
            driver = eyes.open(driver, "Applitools", "Test Web Page", new 
            RectangleSize(1024, 768));
            driver.get("http://applitools.com");
            // Visual validation point #1
            eyes.checkWindow("Main Page");
            driver.findElement(By.cssSelector(".features>a")).click();
            // Visual validation point #2
            eyes.checkWindow("Features page");
            // End visual testing. Validate visual correctness.
            eyes.close();
        } finally {
            // Abort test in case of an unexpected error.
            eyes.abortIfNotClosed();
            driver.quit();
        }
    }
 }
3. after running a test for the first time, it will automatically be saved as a baseline for future test runs. Now run your test again and open Applitools eyes to analyze the changes. For each step either 'Accept New' in    case the changes are expected or 'Keep Baseline' in case the changes reflect a bug.

Advantages of this integration:

• Single page validation replaces hundreds of lines of validation code and hours of manual testing.
• Automatic maintenance – approved changes are automatically propagated to other tests and execution environments.
• Cognitive vision – avoiding false positives, by ignoring differences that are invisible to the human eye.
• Automatically test your app in multiple languages.
• Easily tests complex application pages including dynamic content, size and moving elements.
• It takes the pressure off manual QA and increases the coverage, test faster & more accurately.

This is one of the effective tool to automate Visual Testing in Mobile, Web, Desktop Applications. Happy Testing :) 

Do Makeup to your Jenkins Server

   In continuous Integration Process Jenkins plays a major role in terms of providing a very good platform to perform the task. It gives us the basic requirements such as normal UI, Trend chart. However it is always useful to have comprehensive dash boards, trend charts for better tracking purpose and eye catching UI themes that makes difference. We can also change the email configurations such as Subject lines, body content, including the report into the body of the email etc. For every requirement that is mentioned above, jenkins provides us the plugins. We will tweak those plugins and add the changes that we need. 

HTML Publisher

This plugin is handy with built-in copy to master functionality. Imagine that you have some files generated through your builds that are not actual artifacts. Let’s say logs, execution reports. This plugin lets you to copy them to a master and provide an easy HTML file that will have links to these special files of yours for really quick access.

    To configure this follow the below steps:
1. Click on the Configure option for your Jenkins job.
2. In the post build portion, look for the Publish HTML Reports option and select the check box. See the screen shot below

Fill the path to the directory containing the html reports in the "HTML directory to archive" field. Specify the pages to display (default index.html); you can specify multiple comma-separated pages and each will be a tab on the report page. Finally, give a name in the Report Title field, which will be used to provide a link to the report. By default, only the most recent HTML report will be saved, but if you'd like to be able to view HTML reports for each past build, select "Keep past HTML reports."

Some time at HTML directory, you don't have to specify project name, but have to start one level below. For e.g. if your project name is "ABC" and if HTML files are at "ABC/report-output/html" then you have to specify just \test-output\html\.

3. After saving the configuration, run build once. The published HTML reports are available to view from within Jenkins with convenient links in the dashboard.

Green Balls

Historically Jenkins has used the blue color to denote successful builds. So, all status messages throughout the GUI use blue as the color of success. But if you feel that green is the way to go (Green Hornet, Green Lantern, Gumby, traffic lights, etc) then you can change your preference to green with the Green Balls plugin.

Simple Theme Plugin:
This is a plugin for Jenkins that supports custom CSS & JavaScript.
You can customize Jenkins's appearance (ex. his gentle face on the background) as below:

Navigate to Jenkins > Manage Jenkins > Configure System > Theme
Set URL of theme CSS and JS to URLs of your customized CSS and JS

Some example themes are as below:
http://daniilyar.github.io/Jenkins-themes/themes/canon-jenkins/min/styles.css
http://daniilyar.github.io/Jenkins-themes/themes/canon-jenkins/min/script.js

Wall display Plugin:

A wall display that shows job build progress in a way suitable for public wall displays. Rendering is performed using javascript based on REST API calls, so requires no page refreshes.

After installing the plugin you will get a option on the left hand sidebar.To get it display simply click on the 'Wall Display' link in the sidebar of your project.

Email Extension Plugin:
This plugin allows you to configure every aspect of email notifications. You can customize when an email is sent, who should receive it, and what the email says. 

After installing this, go to the job, which has to configured for mail trigger, Click on “configure”. Go to the postbuild actions, click on “Editable Email Notification”. This will have options like the recipent details etc. Select “HTML(text/Content)” for “Content Type” option. under the Default content text area type the below text as it is though it contains html tags.

Hi All,

 Please find the overall Results of this build as below. You can click on http://10.40.60.186:8080/view/MCS/job/LETSCRETSAutomation/HTML_Report/ for more details.

${FILE,path=”test-output/emailable-report.html”}

Bottom of the line in the above script specifies the path. Change it according to your html path. Remember here, if ur html report depends on any resources like images etc, we have configure them accordingly.

https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

Dash Board Plugin:
This plugin contributes a new view implementation that provides a dashboard / portal-like view for Jenkins.
The configuration is done in 2 parts, selecting the Jenkins jobs to include in the view, and selecting which dashboard portlets to have included in the view.  The jobs that you select are used for all the portlets to draw their information from.