Sikuli

Chapter 1 -Introduction to Sikuli

LEVEL PRACTITIONER

About the Author

Created By: Credential Information:

Automation Testing CoE

Version and Date:

Version 1.0

Introduction to Sikuli: Contents

Introduction:

Sikuli Overview
Sikuli Architecture Supported Platforms Sikuli Integration Why Sikuli?

Introduction to Sikuli : Objectives

Objectives:
After completing this chapter you will be able to understand:
The Overview of Sikuli Sikuli Architecture Platforms supported Sikuli integration with the other IDEs Why to choose Sikuli

Sikuli Overview

What is Sikuli
Sikuli is an open-source research project originally started at the User Interface Design Group at MIT(Massachusetts Institute of Technology - Cambridge). Scripting is done using Python and also with java based languages.

Its further development and maintenance is carried out by the Sikuli Lab at the
University of Colorado Boulder in coordination with the open-source community.

Sikuli is currently maintained and released under the MIT License (Open Source). Sikuli Script automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code.

Sikuli Architecture - The Workflow

Sikuli scripts (.sikuli)

Edit

Sikuli IDE
Use

Images (.png)

Python Execute source file (.py) Import(auto)

Jython API Import

Python Interpreter (Jython)

Java Runtime Environment

Sikuli script

Java API
Java.awt Robot
6

JNI Open CV

Sikuli Architecture

C++ Engine

Sikuli Architecture

How Sikuli Works

Sikuli Script that automates GUI interaction using image patterns to direct keyboard/mouse events uses Jython and Java library. The core of Sikuli Script is a Java library that consists of two parts: java.awt.Robot : Delivers keyboard and mouse events to appropriate locations. C++ engine based on OpenCV : Searches given image patterns on the screen. The JNI(Java Native Interface) connects the Java with the C++ engine and needs to be compiled for each platform.

A thin Jython layer is provided on top of the Java library, for end-users as a set of
simple and clear commands. Therefore, it should be easy to add more thin layers for other languages running on JVM. e.g. JRuby, Scala, Javascript, etc.
7

Sikuli Architecture (contd)

A Sikuli script (.sikuli) is a directory that consists of a Python source file (.py), and all the

image files (.png) used by the source file(Refer the Architecture diagram)
All images used in a Sikuli script are simply a path to the .png file in the .sikuli bundle. The text editor can be used to edit the Python source file. In order to share the scripts on web easily a new HTML file will get generated while saving the script using sikuli. A Sikuli executable script (.skl) is simply a zipped file of all files in the .sikuli directory. Sikuli IDE runs without showing the IDE window if a .skl is seen. If a .sikuli is seen, Sikuli IDE opens it in a source code editor.

Supported Platforms

Mac OSX The current version for OS X 10.6 + Linux 32-Bit on Ubuntu 12.10 64-Bit version on Ubuntu 13.04 Windows On all platforms 32-Bit and 64-Bit Not yet tested on Windows 8 Mobile OS Works with iPhone and Android applications running in an emulator
9

Sikuli Integration

With Other IDEs (Mac and Windows)

Sikuli can be successfully integrated with other IDEs like NetBeans (NetBeans 6.9) and Eclipse (Eclipse Helios 3.6). Integration is needed because
It has more features than sikuli IDE. It can develop packages combining sikuli and java. It has a debugger for the package/scripts.

It can have a versioning system.

The below two plugins are required for Integration

Python plugin that comes bundled with Jython 2.5.1. Python plugin (PyDev) and Jython for Eclipse.
10

Why Sikuli

Why Sikuli
Anything you see on the screen can be automated using Sikuli. It is a free ware and easy to install. Suitable for Multi OS, Multi Browser and Multi Lingual testing. Can be easily integrated with many IDEs using Java. Scripting with sikuli is more easier when comparing with other open source tools. Script less Automation techniques are feasible with Sikuli slides(will be covered on the forthcoming slides)

11

Test your Understanding

What combination of scripting languages are used in Sikuli?

a) b) c) d) Java + Ruby Python + Ruby Java + Python None of the Above

Which of the following browsers are supported by Sikuli?

a) b) c) d) Internet Explorer and Google Chrome Mozilla Firefox and Bing Opera and Netscape All the above

Sikuli can be integrated with Eclipse

a) True b) False
12

Sikuli

You have successfully completed Introduction to Sikuli