Sikuli integration with Selenium

Sikuli – An introduction

SikuliX automates anything you see on the screen of your desktop computer running on Windows/ Mac/Unix. It uses images patterns to identify and control GUI components.

This is useful when we try to automate a page developed using Flex, Adobe or Flash where the html DOM Structure is not defined easily.

That is why it is called as WYSIWYC – “what you see is what you script”

After reading the above introduction, my first impression as a selenium automation engg was “ May be sikuli is something very complex to learn and use in our day to day automation. Unless we spend some good amount of time in it, we can’t think of using Sikuli”

If your thoughts are similar to these, then you are the one to who is supposed to read this..

Let me give you quick step by step start on using sikuli with selenium.

First let us look at the initial installation and set-up procedures.

 1.       Download and Install Sikuli

·         Download sikuli from the following location.

§  http://www.sikuli.org/downloadrc3.html 

§  Follow the steps provided in the link for installation guide. 

·         Once sikuli is installed successfully, one can see the changes in the following environment variables.

·         PATH and SIKULI_HOME 

·         The variable PATH would be appended with “ ~/Sikuli X/Sikuli-IDE/libs”

·         The new Variable SIKULI_HOME is assigned with the value  “~/Sikuli X/Sikuli-IDE”

       This indicates the successful installation of sikuli in the system.

2.       Sikuli Integration with selenium

·         Create a normal java project in eclipse as shown below. Let us name it as SikuliSample.

 

·         Next step is to add the required jars in the build path. To do this, select “ConfigureBuildPath” option as shown below

 

·         Add the selenium jars to the build path.

·         Add “Sikuli-script.jar” which is available in ~/Sikuli X/Sikuli-IDE/libs folder.

·         Add “external class folder” in build path and select the “Sikuli-IDE” folder located at ~/Sikuli X/

The final project created in eclipse would like this as shown here.

With this sikuli selenium integration is done seamlessly. You will now be able to start working on it.

Now that installation and set-up is done… Our next step is to create a sample application.

Problem statement: Verify the Google Logo in google.com webpage.

Prerequisite – sample google logo image file. Let’s assume we have this file located at

C:\Temp\google_logo.jpg

Create a new Class called Sample in the project (SikuliSample) we created above.

Add the following code to this class

public static void main(String[] args){

WebDriver driver = new FirefoxDriver();

driver.get(http://www.google.com);

Pattern image = new Pattern("C:\\Temp\\google_logo.jpg");

Screen s = new Screen();

Match m = s.exists(image);

if(m != null){
       s.click(image);

}else{
  System.out.println("Image not found”);

}

}

In this program, we are trying to verify for the presence of google logo in the Google page and click on the logo if present.

In line #1, which is a normal Selenium webdriver code, initializes a firefox driver.

In line #2, driver opens firefox browser to load the google web page.

In line #3, a Pattern object is created for the image with which we want to compare and verify the webpage.

In line #4, a new Screen object is created. This denotes the screen which is being displayed in the monitor of the system.

In line #5, verify if the image pattern is present in the current screen.

If a Match is present, then go ahead and click on the image.

Otherwise, throw a message on the console screen saying “Image nor found”.

 

Run this program and execute it….

Yes you are done..!!!!