Before getting into the core subject of this blog , it would be good to have an overview on the Selenium web driver and Sikuli to understand how their combination helps in GUI Testing.
About Selenium:
Selenium web driver is the most useful automation tool for web based automation testing. When the script is executed, three parts of Selenium web driver communicate with each other.
- Language level bindings
- Selenium Webdriver API
- Drivers
If you write your test, let’s say in Java and if you use common Selenium API , then the Java binding will send the commands across the above common WebDriver API.
On the other end, there will be a driver to listen to these commands. This driver will interpret those commands and execute them on the actual browser and return the result using the WebDriver API to your code.
About Sikuli:
Sikuli Script is the new way to program anything on your computer. Sikuli is a visual technology to search and automate graphical user interfaces (GUI) using images (screenshots).It automates anything you see on the screen without internal API‘s support. You can programmatically control a web page, a desktop application running on Windows/Linux/Mac OS X, or even an iphone application running in an emulator.
Sikuli also provides a visual scripting API for automating GUI interactions, using screenshot patterns to direct mouse and keyboard events. An user study shows that searching by screenshots is easier and faster than searching through keywords.
Visual scripting also helps to improve interactive help systems like map navigation, bus tracking etc.
Benefits of Selenium and Sikuli integration in GUI testing of E-Commerce websiteS
Sikuli provides user friendly Sikuli-script.jar, which can be used together with Selenium WebDriver. We can even automate Adobe Video/Audio player, Flash Games on website using Sikuli. With simple API, it makes coding easier. This will help you to do GUI testing on an E-Commerce site which has a lot of image based navigations. Using Selenium webdriver incorporated with Sikuli helps us to simulate operations like an end user using the system.
Benefits
- Sikuli programs are written against the user interface instead of an API.
- Generic way for Test Automation on Multiple platforms like Windows/Mac/Linux
- Open source Tool.
- Sikuli is picture driven tool. It captures the images at pixel level for UI verification
- Can be used to automate emulator as well as devices
- Automate all the user operations e.g. Click, type, Drag, Drop, mouse actions etc.
- Using Sikuli with Selenium, we can automate desktop applications.
Creating a new Sikuli Java Project:
The following are the pre-requisites to create a new Sikuli project…
- Any IDE to create project like Eclipse, NetBeans… etc.
- All Selenium related jar files.
- Sikuli-script.jar file
- All the above jar files should get added to the build path.
- Any Screen capturing tool.
Let us try to simulate an end user, using Google website. Here the scenario is to identify the “I’m Feeling Lucky” button and to click it using Sikuli.
- Open Google home page from your browser and Capture “I’m Feeling Lucky” button using any screen capturing tool and save it to your local machine.
Note: Please don’t change the image by highlighting it or by editing. If you do so, then Sikuli may throw an error like “Can’t find image on the screen”.
Type the following code inside the newly created class .Right click on the class select
Run As -> Java Application.
Note: Since Sikuli uses Mouse cursor in real time, please don’t use your mouse until the execution stops. And also don’t navigate to other pages while script is running.
Some Useful Sikuli Methods:
Drawbacks of this tool
- Currently, Sikuli Script operates only in the visible screen space and thus is not applicable to invisible GUI elements, such as those hidden underneath other windows, in another tab, or scrolled out of view.
- And if image appearance is varying in pixel size, will also result in “Find Failed” exception.
- Dealing with moving, animated objects was hard.
- Highly depends on Resolution.
- Dynamic paths are tricky to automate
Conclusion
Selenium integrated with Sikuli is very much useful in automating flash objects. It can be used to automate web and window based applications and for E-Commerce based systems having of images. It is a great tool to play with elements on a screen, based on their visuals.
